At EI, we offer a unique framework to predict/measure learnability, or learning effectiveness, of online courses. With an Analytics plug-in, you can validate predictive learnability with users real-time. In this article, I will outline how you can validate the learnability of online courses.
How To Validate The Learnability Of Online Courses: Case Study
As we know, there is a direct impact of learnability on creating a positive ROI on your training budgets. While the Instructional Design process lays the foundation to create the required learning impact, this approach hinges on several assumptions. Unless validated by the users, we cannot be certain of the learning effectiveness. However, the challenge does not end here. Once we have the user feedback, there needs to be an easy to implement methodology through which the required changes/enhancements/remediation/reinforcements can be applied to the course till we obtain the required learnability. Currently, there are not many frameworks that enable you to validate how the course fared with your users and share the feedback in a manner that you can use to amend your existing learning design.
EI’s framework offers precisely this. You can use it to predict, measure, and validate the learnability of online courses.
Background
At EI, we have been offering custom learning and performance solutions for nearly 15 years now. With a strong core of Instructional Design, our learning methodologies not only meet learners’ expectations but also create the required behavioral change that the business requires.
In Jan’15, we set up a task force to work with a mandate to create a framework that can predict, measure, and validate the learnability of online courses.
A product of our Innovation Lab, our unique framework is now ready. This enables you to:
-
- Evaluate and measure learnability.
-
- Validate the predictive learnability from users through real time surveys.
-
- Use our analytics to enhance/remediate/reinforce the current approach to meet the required learnability.
Through two of my previous articles, I had provided an overview of our framework and a case study showcasing how you can evaluate the learnability of your online courses. Do refer to my previous articles:
This article focuses on how you can use our Analytics plug-in to validate your predictive learnability.
Case Study: How To Validate The Learnability Of Online Courses
I share a case study that showcases:
-
- How you can use our framework to validate the predictive learnability real-time with your users.
-
- How you can take cues from the user feedback and take appropriate action to uplift the learnability to the required level (if there is a gap).
As we know, learning analytics should not be concerned only with gathering data. It should be provided in a manner that you can plan further action. The required action could range from:
-
- Generating further training needs.
-
- Personalization and adaptation.
-
- Intervention.
This is precisely what we have accomplished. We have an Analytics plug-in to capture data from users. More significantly, it helps you analyze data and take suitable action on the reporting.
How Does Our Analytics Plug-In Work?
The Analytics plug-in extends the core capability of our framework to meet your needs to:
-
- Predict learnability (as you develop new online courses).
-
- Measure learnability (relevant for existing online courses).
-
- Validate learnability with users (in real time).
Our Analytics component has been designed to create a robust framework to constantly improve learning, performance, and ROI for the business. It has a data driven approach to address user experience related issues in learning. Specifically, it:
- Supports and extends our learnability metrics.
- Captures actionable user data.
- Predicts and advises on validating/improving the learning effectiveness of an online course.
Objectives of the exercise: The selected course is part of our Suite of 10 online courses for generic Compliance. The course was designed in CY’14 with inputs from Subject Matter Experts so the predictive learnability was in place.
Through the current exercise, we set out to:
-
- Find out how learners were interacting with a compliance course.
-
- Validate how engaging the instructional approach is.
-
- Validate if the learners were taking the prescribed approach as we expected them to.
Activities undertaken in the exercise:
-
- We did a learnability evaluation of the course using our learnability metrics that measure the course against various parameters.
- We selected the critical parameters from the recommendations in the evaluation and created statements that needed to be checked against actual user interaction.
- These statements were tagged into the course as data points that could be tracked using our Analytics framework.
- The course was tested with 10 different users. The parameters of each user were analyzed and reported into the reporting interface to see real time results of user interaction.
Parameters of the learnability validation exercise and the reports (outcome): Here’s a short video that showcases some of the parameters (that were used to validate our assumptions) and a report that highlights where the existing level of the learning effectiveness of our course stands. The video also highlights how you can apply the user feedback to easily update your approach and enhance the learnability further.
How Can You Use This Framework?
This framework will help you:
-
- Make learning a fulfilling experience for learners.
-
- Provide insights on how to make learning more effective and personalized.
-
- Create learning that improves efficiency and predictability in the system.
-
- Track progress of performance for a set period using the data and show incremental gain in ROI.
While we have done extensive testing of our framework and the methodology, we are very keen to partner with organizations in this mandate. Do contact me if you are interested in signing up to measure, predict, or validate the learnability of your online courses.
This article is one of the three articles on how to predict, measure, evaluate, and validate the learnability of online courses. Do refer to my two previous articles to get the complete perspective:
