“Don’t waste your time energy and money working on products and services nobody wants.”
I was asked if I had any thoughts on evaluation this evening, well I guess they wanted to know if I had thoughts on evaluation at any time, it’s just that they asked me this evening, anyway it turns out I didn’t. It’s not that I don’t have any thoughts at all on the subject, it’s just that I can’t pull anything insightful straight off the top of my head at 8pm on a Monday evening.
I decided to give that twitter place a try and see if I could find some inspiration there and you know what, it certainly got me thinking.
First challenge was to define what was I actually asking about – I clarified slightly that I was talking about evaluation of programmes and interventions like leadership/management development programmes or OD interventions.
He also pointed to one of his own blogs about mapping ROI in this case for social media, but applicable widely – good quick read, thx Ian.
His final offering really got me thinking – how could I use one of my favourite frameworks, Alex Osterwalder’s Value Proposition thinking to support evaluation.
This approach would mean evaluation was driven by the customer (learner) and the value they have derived (or perceived to derive) from the learning experience.
At the other end of the spectrum Julie Drybrough reminded me that even big brains struggle with this stuff, pointing out that someone has produced a 104 page evaluation of a Google Leadership Programme that doesn’t mention the learner experience at all.
There are some folk who take evaluation, and it’s cousin Evidence Based Practice very seriously. Thanks to Siobhan for pointing me towards the Center for Evidence-Based Management and the amazing (and slightly intimidating) amount of resource they have available.
Con Sotidis went a stage further and found me a free webinar of the actual Kitkpatricks, presumably talking about the Kirkpatrick model. – Loads of other great stuff there too – Makes you wonder why we bother trying to put anything together ourselves?
Con also mentioned the #OzLearn twitter chat 10 am GMT Tuesday 10th Nov which is in Evaluation. I’m busy unfortunately but Con assures me a Storify will appear on their LinkedIn page
I did do a bit of searching for myself (not much to be fair – will add more later)
I also had a look at The LnDConnect LinkedIn page to see what was there, and found a couple of interesting storifies of #LDInsight chats that were related.
Ken Fee also came up trumps and shared some great resources from Training Zone, which has introduced me to the Success Case Method and Robert O Brinkerhoff. He notes that we often make the mistake of evaluating learning in isolation as one off events, which is like testing the success of a marriage by evaluating the wedding.
No deep thinking yet, just a couple of reflections.
- Evaluation at the end is too late, it has to be an ongoing and iteritive process. What’s the use of a happy sheet at the end of a one day workshop? Really!
- Evaluation should be based on what the learner thinks is valuable and how the learning experience helped them. They are grown ups for gods sakes, who generally know more about their job than you. We should trust them to judge if the learning they have experienced has helped them.
- There is a huge amount of resource out there if you know where to look. This can make evaluation seem big and scary…I think when evaluation is big and scary it is probably taking up too much space in the world.