As we talk more about new learning experiences and incorporating more informal and experiential learning into our efforts, we inevitably end up on the subject of measurement. Everyone gets excited talking about collaboration, games, simulations, virtual reality, etc.; but the excitement comes to a screeching halt once someone asks, “How are we going to measure that?”
It’s not surprising we feel like that. Most organizations really aren’t that good at measuring the traditional learning they’ve been delivering for years. In Brandon Hall Group’s 2016 Learning Measurement Study, just 54% of companies said they were effective or very effective at measuring formal learning. It’s far, far worse when it comes to informal (13%) and experiential (22%). The research also shows that only 21% of respondents are using learning analytics at levels that have an impact on learning strategy and delivery or the business.
(Join me March 8 for a spotlight webinar on learning measurement)
It’s easy to see why when we look at what is being measured. The most commonly used metric for learning is completion of courses. I have absolutely zero idea how the fact that a certain percentage of employees completed a course translates into any sort of result. It is simply a measure of the efficiency of learning, not the efficacy. Numbers two and three aren’t much better: post-course questionnaires and smile sheets, respectively. Learner engagement is important, but the fact that someone liked something doesn’t tell you much about its effectiveness, either. Number four on the list is learner assessments. Now we’re into something with some teeth, but it still is not a reflection of whether or not the learning is having an impact.
It isn’t until number five on the list that companies start getting concerned with actual results – meeting corporate objectives, to be precise. What this tells me is that, in general, the learning function is far more concerned with how well it is delivering learning than it is with what the learning means to the business. The industry of corporate learning is engaged in some serious navel-gazing, where the outcome from learning is learning when it should be behavior and performance.
Don’t get me wrong. There are many companies that actually do get this and are doing it right. In fact, when we look at our data and separate out companies that we consider to be high-performers, something interesting happens. High performers are those companies whose KPIs (revenue, market share, customer satisfaction, etc.) improve year over year. These high-performing companies (HiPOs) do a far better job of measuring learning against actual outcomes than all other organizations. Let’s take a look at a list of outcomes, and see the difference in how many high performers use them as measurements regularly or consistently versus everyone else:
Measurement of Learning Outcomes
In every single category, high performers are far more likely to use these outcomes, sometimes by factors of two, three, and even five! Moral of the story: High organizational performance is correlated with solid learning measurement. I plan to dig into this and more during a Brandon Hall Group Research Spotlight Webinar on March 8.
This brings us back to the challenge of measuring new types of learning. If you look back at the list of outcomes, none of those things are predicated on some specific learning type like a SCORM package. In other words, any type of learning can be measured across these outcomes, as long as you are willing to put in the effort to track it. You have to be able to create a baseline for what these outcomes look like before a learning initiative is rolled out, then measure it again at subsequent intervals. Measuring new learning types is only a challenge if you don’t have a solid measurement strategy to begin with. If you want to break out of the traditional learning box, you are going to have to break out of the traditional measurement box as well.
–David Wentworth, Principal Learning Analyst, Brandon Hall Group
@DavidMWentworth
