The staff development office at Brookhaven offers a wide variety of seminars and workshops—some of them mandatory. Currently the office is working hard to find ways in which workforce training can be made more cost-efficient. One approach to the demand for efficiency is to institute pre-work, where participants show up well prepared, allowing for significant savings in reduced class time. However, as is the case with many learning and development departments trying this approach, Brookhaven was running into pre-work completion rates that were too low for the savings in class time to materialize.
While conceptually sound, pre-work represents a form of just-in-time learning that too often ends up in unnecessary sifting, surfing and page-flipping—boring some participants and overloading others to the point of preventing them from completing pre-work all together. The staff at Brookhaven was not immune to these problems, and as a result the staff development office decided to try a more learner-centric approach to make pre-work happen.
An independent evaluation study was set up with Dr. Michael J. Jacobson, an independent research consultant with 17 years of experience in educational technology and cognitive science, to see if pre-work rates could be brought up to 100 percent. The course used in the study was a mandatory intervention course for supervisors and managers for which pre-work completion rates were running between 50 percent and 60 percent. The course “provides supervisors and managers with the skills to recognize and deal with the problem of substance abuse as it impacts job performance,” said Jacobson.
For the study, instead of the standard printed material, participants used EduCel’s Will-it-Fly? DKTS (Dynamic Knowledge Transfer System) to complete pre-work. This learner-centric system delivers only the chunks of knowledge needed by each individual. It achieves this by acting like a tutor—with real-time assessment functionality and dynamic feedback of chunked and vetted knowledge. The learner-centric approach is critical to the successful execution of the system. To establish the right amount of trust and accountability needed for successful knowledge transfer, associates were informed ahead of time by automated e-mails that participation records would be kept and control questions could be built into the system, but no self-assessment scores would be reported to superiors.
In the study, 47 percent of participants agreed it was very helpful knowing up front that the instructor would not see their front-end assessments. It was also found that a majority of participants preferred the dynamic knowledge transfer system to other methods of preparing for class.
In the end, the learner-centric approach to pre-work resulted in a 33 percent savings in class time. When the time spent by participants on the dynamic system was added back into the equation, Brookhaven was left with net class time savings of 23 percent plus the elimination of the time used by the 50 percent to 60 percent of associates who completed pre-work for this course in the past (no measurement existed for this amount of time). Individual amounts of time spent on the system distributed as a standard bell curve—reflecting the dynamic nature of the learner-centric approach.
“In addition to the benefit of not having to rehash information participants should already know when they arrive in class, the shorter class time represents an incentive for some of the supervisors and mangers who might be reluctant to sign up for this mandatory course,” said Cathy Wehrmann in the staff development office.
The study was designed to have items related to ease of use, learning effectiveness, learning efficiency and helpfulness. According to Jacobson’s report, scores were high in all of these areas, with the survey showing reliability Alpha values ranging from .83 to .93.
“The findings of this evaluation were overall quite positive, both in terms of achieving course instructor objectives and of providing a professional development tool that participants enjoyed using and found efficient and effective,” said Jacobson.
Brookhaven now requires all supervisors and managers to execute the EduCel system ahead of showing up for class.Filed under: Measurement, Technology