Columnists

Published May 2005

Measuring Effectiveness With Learning Analytics

Chris Moore

The term "learning analytics" has been thrown around the training and development industry by everyone from technology heavyweights to analysts, consultants and practitioners. But what does it mean, how does it work, and where's the value? In short, learning analytics means the study of the impact of learning on its learners. One of the most common ways to study the impact of learning is by measuring its effectiveness. Many people use the terms "effectiveness" and "impact" synonymously when it comes to measuring learning, but there are differences.

The origins of measuring the effectiveness of learning relates to its pedagogy-seeking to understand whether learning effectively meets its original design objectives. Furthermore, CLOs seek to understand if learning is effective based on the manner in which it is delivered. Studies by Richard Clark have shown that there is little difference in effectiveness between traditional and technology-delivered learning, so the real focus in measuring effectiveness returns to its pedagogy.

There are good arguments that position learning effectiveness as a learning measure, not a business measure. However, in the business of delivering learning, the line between the two becomes less clear. What learning executives are really after is measurement of the impact learning has on the learner. Effectiveness plays a key part here. If the learning is not properly designed and aligned to the business goals of the organization, the impact on the learner is less likely to produce the desired results. Learning analytics can help CLOs measure and monitor both effectiveness and impact, as well as the overall goals of their learning organizations.

Why Learning Analytics?
Analytics is the breakdown or decomposition of a system through logical analysis. In the case of CLOs, that system is learning and the myriad data points that accompany it through its evolutionary phases of planning, design, development, delivery, implementation, maintenance and retirement. Each phase introduces numerous dimensions, such as learners, instructors, resources, objectives, actions, processes and outcomes. The logical part of analytics provides the ability to ask questions about the learning system and the relationship between the various dimensions. Keep in mind that "learning system" refers to the whole learning ecosystem, including the business systems it touches-not just a single system like a learning management system (LMS) or learning content management system (LCMS).

Why do you need analytics at all when you've invested in an LMS or LCMS? Don't these systems have turnkey and custom reporting capabilities? Yes, but most of these systems are data-rich but information-poor. They collect and store thousands of data points of learning activity and process, but rarely present the data back in an informative way that allows business decisions to be made. Most of the reports included in the system are activity-based. These types of reports are valuable because they help CLOs manage the learning environment, but they don't help them lead.

1 2 3 4 Next Page »

blog comments powered by Disqus

Executive Search

Senior Manager, Global Learning & Talent Development
11/19/2009
Deloitte Touche Tohmatsu (DTT) is an organization of member firms devoted to excellence in providing professional services and advice. We are focused on client service through a global strategy executed locally in nearly 150 countries.

Director, Leadership & Organizational Development Parkland Health & Hospital System
10/26/2009
Parkland Health & Hospital System (www.parklandhospital.com) located in Dallas, Texas has been voted one of "America's Best Hospitals" by U.S. News & World Report for 16 consecutive years and recently named one of the "Top 100 Hospitals to Work For" by Nursing Professionals Magazine.

The World Bank Knowledge and Learning Coordinator Washington, DC
12/22/2008
The Latin America & Caribbean Region (LCR) of the World Bank serves over 30 countries, mostly middle-income which, despite having middle-income economies, still struggle with pockets of poverty and high level of inequalities.