The Challenge
One of the biggest challenges companies face when it comes to delivering impactful learning is an inability to measure and demonstrate training effectiveness and its impact on business outcomes. In other words, the problem is not just that companies cannot adequately measure the effectiveness of their learning, but that as a result, they cannot improve their programs. In Brandon Hall Group’s Transforming Learning for the Future of Work study, the number three challenge facing L&D teams is that “we don’t know how to measure learning well enough to ensure the future skills development we need will be achieved.”
Organizations are just not that mature in their approach to learning measurement. Most are heavily focused on completion rates and smile sheets, which are not providing them with enough insight into how learning improves the business or helps it achieve its goals. Brandon Hall Group research finds that only 27% of companies say their learning strategy includes a framework to measure success.
The Rethink
The silver lining to this is that this represents a recognition that measurement is not some singular endpoint in the learning lifecycle. Measurement must be an integral part of learning from the outset. Otherwise, learning has no alignment with business outcomes and L&D teams are flying blind. This is why so much measurement is focused on grades and completions. While this data is important, it provides no actionable insights on its own.
Ultimately, it is about shifting the focus of learning measurement from efficiency — enrollments, completions, grades, etc. — to efficacy. Measuring how many people finished a course is meaningless unless you can show that those people are behaving or performing differently than those who did not. The outcome of learning cannot simply be that learning occurred. It must be behaviors that drive performance.
The Recommendation
What Brandon Hall Group has to Say About EI’s Solution
Brandon Hall Group Smartchoice® Platinum Preferred Provider EI is leveraging its solutions to help its clients accelerate their learning measurement maturity. As a strategic partner, they help their clients build a measurement mindset that not only enables better measurement but also helps drive maximum return.
EI’s Training Needs Analysis (TNA) framework helps companies build a measurement foundation. This includes two key elements:
-
- Business Case. The purpose, description, and expected outcomes of the program must be defined
- Performance Metrics. The specific metrics that are associated with the expected outcomes must be defined, as well as the ways in which they will be measured.
They work with L&D teams to help engage business leaders to identify the key performance indicators for the business. Learning metrics can then be aligned with these KPIs to ensure learning is continuously focused on driving business outcomes.
Measurement is often overwhelming for L&D teams, given the scope of what they are trying to measure. There are many models and frameworks available to help but choosing the right one can be its own challenge. EI helps clients pick the model that will work best in their environment, whether it is an existing model or a custom framework. They also provide a toolkit for getting jumpstarted on better evaluation:
Step 1 — Redefine what learning means. To gain real insight, there must be a mind shift that goes beyond the idea that learning is simply knowledge acquisition. In order for learning to have taken place, it means that learners have:
-
- Acquired new information
- Practiced a skill or behavior
- Applied that knowledge/skill/behavior on the job.
Step 2 — Focus on the business, but don’t ignore the learner. While organizational goals are important, they must be aligned with learner needs and expectations at the outset. Because each learner is different, it might take different learner journeys to achieve the same business outcomes. Factoring in these needs makes it possible to:
-
- Leverage learners’ past knowledge and experience.
- Offer learning opportunities that best align with what learners want and need
Step 3 — Identify relevant and objectively measurable metrics. There needs to be a combination of related and aligned business and L&D metrics associated with learning programs. Strong collaboration between the L&D team and business stakeholders will help define how impact will be measured. Positive L&D outcomes alone may give a false sense of effectiveness. Key performance indicators are required to determine impact. Effective collaboration with business stakeholders means:
-
- Leveraging L&D expertise to articulate how learning can impact performance
- Understanding the metrics that are important to business leaders
- Focusing on objectively quantifiable metrics
- Determining the value of measuring a program’s effectiveness
- Reporting results honestly, even when the outcomes are poor.
Step 4 – Identify measures to sustain and maximize impact. There are several strategies for ensuring learning’s impact can be sustained. This includes focusing on both L&D and business metrics to set the foundation. Additionally, an appropriate evaluation model should be selected. This will provide clarity around what will be measured, how it will be measured, and what will be done based on those measurements. L&D teams should also spot-check the metrics periodically to see if things are moving in the right direction, without waiting for a big, final measurement event.
Ultimately, this approach to measurement will help organizations build and improve programs that continuously sustain performance. By tracking what is working, efforts can be focused on doing more of that, and retooling areas that are not producing impactful results. These are the kind of insights that completion rates alone can never deliver.
David Wentworth, Principal Analyst, Brandon Hall Group
