Evaluating and Assessing Modern Blended Learning Outcomes

Posted by Charles (Chip) Dye on Sep 26, 2017 1:15:00 PM

31944785_s.jpgIn discussing the concept of “blended learning,” we have previously stated that, “modern blended learning is selecting the most appropriate delivery technology based on learning objectives, with consideration of time, place, and space.” So it is, that as we consider how to assess a blended learning solution, we need to look at that statement and look at the objectives that required a training solution in the first place – finis origine pendet – “the end depends upon the beginning”.

When a blended solution is implemented, it is directed at providing learners with the opportunity to master/complete a particular set of individual learning objectives. To assess such a training approach, many will stop there, and seek to assess against the individual objectives, absent any consideration of the delivery method or specific organizational outcome that drove the individual learning objective treatment in time, place, or space.

To illustrate, let’s look at an example:

Mel is an insurance adjuster for Magnanimous Insurance Company, Inc. Magnanimous has just released a mobile technology solution that allows adjusters in the field to fill out the requisite claim documents and append pictures and related media to expedite the claim for the customer. To support the roll-out of this technology, Magnanimous provided self-paced (time) training online via mobile tablet (place), to allow adjusters to practice in actual field situations (space) while using the old method to actually document the claim. In addition, they held weekly live virtual seminars to allow debrief of feedback from the field and to solicit any input on making the mobile solution better.

QUESTION: How do we assess this training outcome?

ANSWER: Look at the desired organizational outcome, and then evaluate how that objective is supported by learner outcomes. Magnanimous Insurance wants to streamline the submission and documentation of claims – this lowers costs – but as they are in a highly regulated industry, Magnanimous also needs the documentation to be complete and correctly submitted. Using the old system, the error rate was 4.7%. When stated this way, the assessment and evaluation of the learner and the blended training solution becomes fairly apparent – use the operation of the mobile application to evaluate actual performance by Mel and his peers, and compare the error rates with those from the old way of submitting claims.

It is important to note in this example that the assessment and evaluation approach needs to be considered long before the training solution is implemented. Like most traditional development models (such as ADDIE), a learner assessment (and we would add organizational evaluation) in the training solution must be planned and designed before actual implementation.

“OK, I get it,” you might be thinking, “But how is blended assessment any different?” Great question!

Blended Assessment is MORE Capable

The key to understanding the difference is in the varied learning environment and capabilities at your command in a blended solution. Rather than relying on delivery in the same treatment method(s) used to deliver the training content, you have largely unfettered use of all blended methodologies to assess, and in addition, can draw from latent data sources (say, for example, mis-filings of insurance claims in the mobile application, or user latency in filing out electronic forms) to determine learner progress in achieving desired outcomes. Moreover, with many blended treatments and environments, there is an inherent level of fidelity to real world performance that a traditional method of instruction or assessment can’t provide.

To conclude, the assessment of the learner can be integrated with an evaluation of the blended training solution to provide insight in both learner progress and the achievement of organizational outcomes. With a variety of technologies and strategies to assess the learner, don’t feel constrained to stick with the old multiple-choice-test-and-effective-instrument solution – leverage the capabilities of the modern classroom to take advantage of passive data collection and tools!

Topics: Assessment, Modern Blended Learning