Evaluating learning – where to start?
It’s easy to forget to do it, and some people even dread it, but evaluation is critical to success. You need to understand what works well in a piece of training, but more importantly, what doesn’t, so you can continue to adapt and improve.
A common approach is to use the Kirkpatrick’s Four Levels of Evaluation combined with the Phillips ROI Model.
If they don’t mean much to you at the moment, read on and all should become a lot clearer.
Let’s start with the Kirkpatrick’s Four Levels of Evaluation.
The model suggests that effective evaluation should take place at four different levels: Reaction, Learning, Behaviour and Results:
Level 1: Reaction
The first level is about gauging learners’ thoughts or feelings about the training, both positive and negative, and whether they feel the training will help them with their jobs. This is normally done with a survey or questionnaire, or less commonly through participant interviews.
Level 2: Learning
This is all about understanding whether participants have gained the desired knowledge or skills from the training. Testing or questioning before and after can be very effective here.
Level 3: Behaviour
At this level, we need to assess whether the training has impacted behaviour. In other words, whether the participants actually go on to use their newly acquired skills or knowledge in their day-to-day jobs. There are various ways we can do this, but on-the-job observations or evaluations from line managers work well.
Level 4: Results
And lastly, we look at how any behaviour changes have impacted the business. For example, if you improve sales team’s performance, then sell more products, you increase revenues. Or if you make managers more effective, you’ll have happier employees and reduce staff turnover.
Level 5: The Phillips ROI Model
The Phillips ROI Model builds on the Kirkpatrick’s model to offer a fifth level. It helps organisations to compare the overall cost of training to its monetary benefits. In other words, it’s a cost/benefit analysis to determine whether money invested in training has produced measurable returns.
Though it’s a useful measure, it’s notoriously difficult to do as you need to isolate the benefits from training, while eliminating any other (non-training) contributing factors.
In fact, it’s not always relevant to go through all five evaluation levels for all training programmes. For smaller pieces of training, it may only be useful to evaluate at level 1. However, if management teams want to see exactly how much a training programme is impacting a drive to increase revenues or reduce costs, figuring out the ROI is the way to go.
Whichever route you go down, it’s important to think about evaluation early on in the programme development phase. By working backwards from the business impact you want to achieve, you can create effective training from the offset.