One of the most popular management maxims is, “If you can’t measure it, you can’t manage it.” Usually and apparently falsely attributed to management guru Peter Drucker, it nonetheless pops up in management speak and business circles far and wide.
Regardless of source, chief learning officers have taken it to heart. Ever since Don Kirkpatrick’s eponymous learning evaluation model roared out of Wisconsin in the 1950s, learning professionals have been busily standardizing, collecting and analyzing a host of learning outputs from smile sheets and course assessments to behavior change and productivity measures.
But widespread practice hasn’t necessarily translated to effective management. Many learning organizations continue to measure outcomes of learning activity and learner satisfaction while neglecting broader business performance results such as sales or product quality.
According to a survey of the Chief Learning Officer Business Intelligence Board, an overwhelming 93 percent of learning organizations either currently or plan to measure employee response to training (Figure 1). Only 50 percent currently measure learning’s impact on business performance and even fewer (36 percent) extend the effort to specific results like sales.
The Chief Learning Officer Business Intelligence Board is a group of 1,500 professionals in the learning and development industry who have agreed to be surveyed by the Human Capital Media Research and Advisory Group, the research and advisory arm of Chief Learning Officer magazine. This survey was conducted from June to July 2017.
That disconnect between measurement efforts and business outcomes may help explain learning professionals’ general dissatisfaction. More than half report being unhappy with the state of their learning measurement efforts (Figure 2).
The reasons vary. Many learning teams lack data expertise or are short of resources or desire to pursue sophisticated measurement efforts. Some learning organizations are simply not expected to produce credible business data.Capability is undoubtedly a factor. According to the BIB data, nearly 70 percent of learning organizations plan to increase their analytics capacity in the next two years.
Technology presents another barrier. The rise of Big Data, the popular term for large sets of structured and unstructured data, has made sophisticated collection and analysis tools critical to success. Only 14 percent of learning organizations report they have the technology needed to collect, integrate and analyze
data across multiple HR systems. Forty-four percent report they are able to do that to some extent and 42 percent not at all (Figure 2).
Despite the challenges, reporting of outcomes is widespread in many organizations (Figures 3 and 4). But many of those reports are either manually generated or automated from the learning management system (34 and 26 percent). Fourteen percent have no formal metric reporting in place.
Among the top metrics reported to executives are general training outputs, such as number of people trained or learning hours delivered, and satisfaction with the quality or availability of learning. Business impact and employee performance data are lower on the list and specific measures like Net Promoter Score and ROI fall even lower.
As this data indicates, what learning professionals are measuring is not what is being managed, at least when it comes to broader business outcomes.
Mike Prokopeak is vice president and editor in chief of Chief Learning Officer magazine. Comment below or email editor@CLOmedia.com.
Filed under: Measurement, TechnologyTagged with: big data, Chief Learning Officer Business Intelligence Board, chief learning officers, Don Kirkpatrick, education technology, Human Capital Media Research and Advisory Group, technology
