By Susan Camberis
Editor, Training Today
Measuring learning impact is critically important yet knowing how to begin can be a challenge.
ATDChi’s August webinar featured Olga Polyakova. Olga is the Learning Analytics lead for BCG North America and a former ATDChi Board Member. Olga shared her experiences with measuring learning impact in professional services.
Within BCG, career development is handled by career advisors not line managers. While the vast majority of training is mandatory, it can still be difficult for team members to find time to attend. Talent rotates on and off new projects every two to four months. Everyone is evaluated on every project, but evaluation data is not connected to learning.
“The best thing you can do is build proof-of-concept,” said Polyakova. To do this, Olga initially focused her efforts on measuring the impact of a three-day off-site training that BCG offers regularly. By combining delayed-in-time data with compelling personal narratives and factors that enhance or impede sustained impact, Olga was able to create an impactful and sustainable model.
Here are 5 code-cracking take-aways:
1. “Think, then do.” When Olga first began enhancing learning analytics at BCG, she was also working on her Masters in Training and Organizational Development at Roosevelt University. According to Olga, this was a “magic time” activity – an add-on to her already packed schedule, so maximizing impact was the only way to be successful.
2. Keep it simple. Olga’s objectives starting out were simple: better data (e.g., more outcomes-focused ways to measure feedback), new data (e.g., over-time quantitative and qualitative impact analysis), and better process (e.g., increased automation). Olga helped BCG develop a two-pillar evaluation framework based on feedback and impact that she believed would resonate with the organization’s culture. By analyzing data collected at 3-month and 6-month check-ins, the team identified the most applied learning’s as well as behaviors with lower uptake.
3. Focus on the value proposition. By improving the data being collected, focusing on application data, and leveraging data to improve the overall employee experience at BCG, Polyakova was able to make a compelling case for how the data could deliver benefits for the learning function and the business.
4. Streamline impact analysis. Establishing impact analysis typically involves three steps: Design, Execute, and Report. Execution is what can seem overwhelming, but it doesn’t have to be. One of the best demonstrated practices that Olga shared was to complete the Design and Report steps at the same time, so that you can show your customers what they will get the for data you are asking them to provide.
5. Quantify results. “Even for the most intangible trainings, try to quantify results and support with quotes,” shared Polyakova. If you can quantify time or cost savings estimated by participants, you can extrapolate broader savings and impact for the organization. In this case, personal narratives were analyzed for quotes to illustrate impact on performance. The team also discovered that the data they were collecting provided an amazing source of ideas and suggestions for improving the wellbeing of BCG team members and included peer advice that could be shared with future program participants.
As with any complex project, getting started is key.
“You really just need to start and do more than was done before,” said Polyakova.
Want to learn more?
Follow Olga Polyakova (https://www.linkedin.com/in/polyakova-olga/) and BCG
(https://www.linkedin.com/company/boston-consulting-group/) on LinkedIn.
If enhancing learning analytics is one of your objectives this year, consider joining ATDChi for a full-day workshop with October 15th with Ken Phillips focused on predictive learning analytics. To learn more, visit: https://atdchi.org/event-3276628.