CLOs know that metrics and measurement programs are more successful when they come from a deeper business need or reflect the kind of business thinking that already pervades the culture. Programs that are imposed artificially from the outside and that are not expressions of the way in which the organization already works may either fail or end up measuring the wrong kinds of things.
Kevin Wilde, chief learning officer for General Mills, notes that when he set out to find the right ways to measure the impact of learning on the performance of General Mills, he drew on what was distinctive about the company. Early on, said Wilde, “I realized that, to build the credibility and relationships with senior executives in the company, I needed to speak their language.” Realizing that type of data is a key lever in the company, Wilde set out to determine the best data to show that learning programs were having an impact on employee commitment and performance. The employee survey turned out to be a particularly effective tool. “I have leveraged the heck out of these surveys because that’s the language our leaders appreciate. It helps to show the numbers. So when we doubled the size of the company two years ago with the Pillsbury merger, I used employee survey data to help us understand some of the people issues on both sides. We then provided the right response to smooth many of the merger speed bumps we found. Subsequent surveys indicated solid progress,” he said.
The main point, said Wilde, is that learning professionals need to link their work with the business strategy, then provide differentiating value and periodically show the impact with long-term metrics that speak powerfully to decision makers.
At Wachovia, the metrics work being done by CLO Ron Garrow and his team was driven by a real business need: the merger of Wachovia and First Union in 2001. Wachovia CEO G. Kennedy Thompson set an extremely high goal for the new organization following the merger: zero customer impact. The training that was needed to ensure that employees could perform in a new environment with no service repercussions was substantial. It involved about 1.6 million hours so that the workforce could learn new products, processes and systems and ultimately, so the organization could exceed customers’ expectations for servicing through the entire merger integration period.
In order to meet the goal of zero customer impact, said Garrow, he and his team worked with line-of-business training teams to develop and administer “readiness assessments” before each region or line of business did any type of conversion based on the new processes and systems. “That readiness involved not only Level 1 evaluation—evaluation of the participant and employee satisfaction with the training— but also measurements of their comprehension and retention and even of how well they were able to apply that knowledge. So the business circumstances we were dealing with forced us rather quickly into some solid Level 2 and 3 evaluation data,” Garrow said.
The measurement program put in place to meet the immediate business need of the merger then became the basis for what Wachovia called its “Assessment, Measurement and Evaluation” or AME philosophy. Garrow and his staff have identified AME champions within each of the learning teams that are part of the learning organization. The AME champions are in the process of going through an ROI certification. Wachovia is already seeing some benefits and demonstrated results from this initiative.
For a new executive leadership program the company is developing, Garrow and staff assigned executive coaches to each of the participants. By interviewing these coaches and the subordinates of the program graduates, Wachovia intends to collect Level 2, 3 and even Level 4 data about how the program has affected the performance of these leaders and the company as a whole. Garrow said, “All in all, this data is really going to help show us exactly what these graduates are doing differently as a result of that educational experience.”
These stories underscore the importance of going beyond the “right” external measurement methodology. What’s needed is an approach that operates out of a deep understanding of the business levers of your company and uses relevant business events to drive an approach to measuring the business impact of learning that resonates with executive leadership.
Jeanne C. Meister is vice president of market development at Accenture Learning. Comments on this article can be sent to Jeanne at firstname.lastname@example.org.Filed under: Measurement