In Practice

Published February 2008

The Regence Group: Blended Measurement

Chris Moore

Created in 1995, The Regence Group is one of the largest affiliations of health care plans in the U.S. Pacific Northwest region, collecting $6.5 billion in annual premiums and serving nearly 3 million members across Oregon, Washington, Idaho and Utah.

Customer satisfaction is a top priority at the company, and executives realize high satisfaction levels are directly linked to the readiness of their employees. Ongoing learning initiatives with appropriate and timely end-of-course surveys help the organizational development (OD) team at Regence collect learner feedback and ensure that training is both aligned with the business strategy and effective in transferring the required knowledge and skills.

In its surveys, Regence finds it best not to rely solely on Likert scale questions to characterize the success for its training initiatives. Using assessment software, OD adds a series of open-ended questions into its surveys in an effort to solicit anecdotal feedback from
its audience.

That’s not to say that Likert scale questions don’t have their time and place at Regence. Most students can and will complete them at the end of a course with little effort. However, while Likert-based questions are easier to quantify, inconsistencies can arise because of the minimal thought needed to “shade the box.” In contrast, open-ended questions require more thoughtful, methodical and honest input from employees. Regence employs a clever approach to quantify and automate its review of its electronically captured feedback.

“We gather and interpret quantitative and qualitative data for regulatory and ethical compliance, business-process improvement, professional development, training in the right topics and user readiness for enterprisewide systems,” said Jim Baker, senior e-learning specialist. “We gather metrics and narrative responses through the pull-down list, survey matrix, customizable Likert scales, ranking order, multiple-choice and true-false and yes-no, Flash, multiple response, fill-in-the-blank and essay questions. Except for the essay questions, where we already gather anecdotal information, we add a comment box to other types of questions. We share with management the metric fluctuations and trends in employee opinions.”

At Regence, anecdotal feedback is analyzed both quantitatively and qualitatively to pinpoint topics that are of particular interest or concern.

Quantitatively:

• Since people will spend time writing about something they care about, a pattern of long responses to a given question indicates a high level of interest in it.
• Large quantities of open-ended responses to a given question help identify areas
of interest.
• Scanning the number and length of responses can quickly pinpoint the most compelling issues.

Qualitatively:

• Comments are exported to Microsoft Word where there’s a search for emotive words like "never" and "always." These words often reflect strong feelings about a subject, e.g., "It was never this way before!"
• Strongly worded comments that match up with low ratings on the Likert scale can help prioritize issues for management.
• A representative selection of comments is provided at the end of statistical reports. The reasons for low and high scores on a survey are often revealed in the text responses.

Tuning in and listening to employee feedback, whether audibly or electronically, is vital to the ongoing success of learning initiatives at Regence. Its keen sense of the words and phrases that imply value and meaning to its employees gives the company insight into
continuous improvement of OD’s initiatives, ensuring they provide equal benefits to their clients and their employees.

Chris Moore is president of Zeroed-In Technologies, creator of Zeroed-In Dashboards, solutions for visualizing, measuring and executing performance strategies.

blog comments powered by Disqus

Executive Search

Senior Manager, Global Learning & Talent Development
11/19/2009
Deloitte Touche Tohmatsu (DTT) is an organization of member firms devoted to excellence in providing professional services and advice. We are focused on client service through a global strategy executed locally in nearly 150 countries.

Director, Leadership & Organizational Development Parkland Health & Hospital System
10/26/2009
Parkland Health & Hospital System (www.parklandhospital.com) located in Dallas, Texas has been voted one of "America's Best Hospitals" by U.S. News & World Report for 16 consecutive years and recently named one of the "Top 100 Hospitals to Work For" by Nursing Professionals Magazine.

The World Bank Knowledge and Learning Coordinator Washington, DC
12/22/2008
The Latin America & Caribbean Region (LCR) of the World Bank serves over 30 countries, mostly middle-income which, despite having middle-income economies, still struggle with pockets of poverty and high level of inequalities.