As we explore new learning experiences and incorporate informal and experiential learning into our efforts, we inevitably end up on the subject of measurement. Everyone is excited about collaboration, games, simulations, virtual reality etc. but the conversation grinds to a screeching halt as soon as someone asks, “How are we going to measure that?”
That’s not surprising; most organizations really aren’t good at measuring the traditional learning they’ve already been delivering. In Brandon Hall Group’s 2018 Learning Measurement Study, only 51% of the companies said they were effective or very effective at measuring formal learning. It’s far, far worse when it comes to informal (19%) and experiential (29%) learning.
It’s easy to see why, when we examine what’s being measured. The most commonly used metric for learning is completion of courses. But that just measures the efficiency of learning, not the efficacy. Numbers two and three aren’t much better — post-course questionnaires and smile sheets, respectively. Learner engagement is important but the fact that someone liked something doesn’t tell you much about its effectiveness. Number four on the list is learner assessments. Now we’re onto something with a little bit of teeth, but it still is not a reflection of whether the learning has impact.
It isn’t until number five on the list that companies show concern for actual results – meeting corporate objectives, to be precise. What this tells me is that in general, the learning function is far more focused on how well it delivers learning than it is with what learning’s impact is on the business.
Don’t get me wrong: Many companies actually understand the difference and are getting it right. In fact, when we look at the data, especially high-performing companies, something interesting happens. (High performers are those companies whose KPIs — revenue, market share, customer satisfaction, etc. — improve significantly year over year.) They do a far better job measuring learning against actual outcomes than other companies.
Let’s look at a list of outcomes to see how many high-performing organizations (HiPOs) use them as regular or consistent measurements versus everyone else (click to enlarge):
In every category, high performers are far more likely to use these outcomes to measure the effectiveness of their learning. In other words, high organizational performance is linked to solid learning measurement.
This brings us back to the challenge of measuring new types of learning. If you look back at the list of outcomes, none are predicated upon some specific learning type like a SCORM package. All learning can be measured across these outcomes as long as you are willing to track it. You must be able to create a baseline that shows what these outcomes look like before a learning initiative rolls out, then measure it again, repeatedly.
Bottom line: Measuring new types of learning is only a challenge if you don’t have already have and use a solid measurement strategy.
–David Wentworth, Principal Learning Analyst, Brandon Hall Group
@davidmwentworth
