Features

Published February 2010

Validation by Metrics

Tom Kelly

Busy metrics are irrelevant and uninspired — at least to executives outside the training function. Though hours spent in training, videos watched and “butts in seats” reports may still allow training leaders to get some indication of efficiency, executives today require a new framework focused on effectiveness.

Long gone are the days when rigid reports pulled from registrations and tracking technology would suffice. You want your good work to be validated, right? Well, old-school metrics from just one department — the training department — haven’t worked in the past and aren’t going to make it happen.

These activity-focused measurements, called busy metrics, are often still valuable to training departments, individuals and compliance officers, but as learning executives align with the organization’s greater objectives, they must evolve to relevant, agile business metrics. Looking at the changing learning landscape, metrics will continue to be problematic until learning practitioners embrace measurement systems that are informative and adaptable to the entire enterprise.

“Our industry is inundated with activity metrics, and we think that as long as we see how much we are doing, we must be doing something right,” said Doug Harward, CEO of Training Industry Inc. “The real advances in measurement theory will help training managers understand the difference between activity and achievement.”

The Crutch

A common complaint about learning management systems (LMSs) is their lack of reporting flexibility. In a survey of LMS customers for the Bersin& Associates study “Learning Management Systems 2009,” 45 percent of respondents cited reporting capabilities as the No. 1 challenge with their current LMS. Reporting was also the highest ranked driver of dissatisfaction in the study.

According to David Mallon, senior analyst for Bersin& Associates, the best systems in this area are able to capture and report on all completion data — all LMSs do this, but some struggle with providing the same level of support for imported content, such as SCORM e-learning versus native activities; aggregate data by manager or other user grouping — most LMSs do this; compute completion percentages; and generate reports that show exceptions (e.g., who has not completed, who is more than 30 days late and so on) by group and manager.

“LMS providers cannot account for every customer’s reporting needs, so built-in stock reports will only be so useful,” said Mallon.

Traditional LMS metrics are generally made up of isolated data focused on training delivery or deployment. LMSs were created to address the needs and reporting metrics of training organizations. However, because they are training-focused technologies, they are not integrated with the functional performance elsewhere within the enterprise.

1 2 3 4 5 Next Page »

blog comments powered by Disqus

Executive Search

ESI International Director, eContent Strategy
01/14/2010
The Director, eContent Strategy is responsible for providing ESI�s executive team with strategic-level direction to implement alternative blended learning delivery formats to our worldwide client base.

Senior Manager, Global Learning & Talent Development
11/19/2009
Deloitte Touche Tohmatsu (DTT) is an organization of member firms devoted to excellence in providing professional services and advice. We are focused on client service through a global strategy executed locally in nearly 150 countries.

Director, Leadership & Organizational Development Parkland Health & Hospital System
10/26/2009
Parkland Health & Hospital System (www.parklandhospital.com) located in Dallas, Texas has been voted one of "America's Best Hospitals" by U.S. News & World Report for 16 consecutive years and recently named one of the "Top 100 Hospitals to Work For" by Nursing Professionals Magazine.