Cultivating an Analytical Mindset in L&D

One of the key qualities of anyone involved in L&D is having an analytical mindset. It is crucial for the continuous improvement of learning programs, which means it’s crucial for the continuous improvement of business results. Almost everyone in L&D will tell you they’re passionate about continuous improvement, but how many are dedicated to cultivating an analytical mindset? How many are evaluating programs so that they can be improved in future? How many are assessing learners continuously throughout the program and making adjustments to ensure the most effective learning experience and knowledge transfer?

When it comes to virtual and hybrid learning programs, often our only feedback comes from “smile sheets,” the surveys at the end of class to see what learners thought of the program.

These, however, rarely help guide our professional development, especially when only five out of thirty learners actually filled it out. If we want to assess how things are really going in a virtual facilitation setting, we must incorporate ongoing assessment strategies that provide valuable insights into the effectiveness of training methods and materials.

By moving beyond end-of-session surveys and embedding various types of assessments throughout the learning journey, we can more accurately measure knowledge transfer, adapt our facilitation techniques, and enhance learner engagement. This approach of ongoing assessment leads to higher content retention and more effectively meeting learning outcomes, which elevates the standards of our training programs, and ultimately elevates the standards of the entire L&D field.

Why do Analytical Mindsets Matter?

At InSync, we provide high-quality, effective learning experiences by championing continuous assessment and engaging design. We have a reputation for excellence in virtual learning because we ensure all our programs are not only interactive, but also impactful. And that impact is not just something we hope for; it’s something we’re intentional about at every step of the learning journey. We know that you, as an L&D professional, are also aiming for interactive and impactful learning in your organization. By adopting an analytical mindset and following the tips shared in this article, you too can monitor learner progress in real time, make data-driven decisions, and tailor your training methods to better meet the needs of your learners. You can improve learner outcomes, use resources more efficiently, and achieve a higher return on investment from your training programs.

Before we consider how to cultivate analytical mindsets in ourselves and our teams, let’s consider how to incorporate continuous assessment in learning programs.

First, we need to stop thinking that assessment only happens at the end of the training program. We should be assessing in multiple ways and at multiple times throughout a program. Participants in any training learn in various ways; therefore, we should use many different approaches to assess their learning along the way.

Types of Assessment

Assessment in virtual learning environments can be categorized into two primary types depending on when the assessment takes place and what the purpose of the assessment is: formative and summative.

Formative assessment occurs during the learning process and includes activities like quizzes, diagnostic tests, and in-class assignments that inform facilitators about learners' progress. These assessments help adjust the instruction to meet learners' needs.

Summative assessment, on the other hand, takes place at the end of a learning unit to measure knowledge transfer and evaluate the effectiveness of the facilitation and learners’ understanding.

Remember, it is not enough to survey the learners at the end of a lesson to determine if virtual classroom teaching is effective. Incorporating assessment into the virtual classroom will increase interactivity and engagement, which will lead to much higher transfer of content and retention. Implementing this assessment and the benefits it brings is the responsibility of the designer and the facilitator.

Engaging Design for Effective Evaluation

Designers can incorporate formative assessments for virtual learning programs by intentionally designing assessments throughout the program. Steps could include designing pretests and pre-surveys to evaluate the learners’ skillset before coming to the virtual classroom. They could include building periodic quizzing techniques into the live classes with plenty of supporting information to help the facilitator debrief them. They could also include practical activities and role plays that allow the facilitator to observe the learners practicing their new skills.

Designers may be more traditionally involved in summative assessments. This type of assessment typically reflects the facilitator’s ability to communicate the information effectively and the learners’ ability to take the information in.

So how can virtual learners be successfully assessed in a summative way?

Written Assessments 

Most virtual classroom platforms include some sort of testing feature that you can use either in the classroom or outside of the classroom to measure knowledge transfer. Traditionally, written assessments include quizzes and tests. Research some learning apps you could integrate into your virtual classroom to allow you to test your learners in fun and engaging ways.

Performance Assessment

This type of assessment requires learners to perform a task or skill as a means of showcasing their ability to put the content to work. Performance assessments evaluate complex cognitive processes as well as attitude, social skills, and the ability to utilize the technology. Consider including performance assessment methods such as these:

• Debates: Learners take a stance on a topic, discuss it in small (breakout) groups, and defend their position to the entire group.
• Teachbacks: During a teachback, learners select a piece of content, teach it back to the group, and accept feedback from the cohort and facilitator (if appropriate).
• Project-Based Assessment: Learners create materials to represent the topic and showcase understanding and application.
• Portfolio Assessment: As a way to review the progress of the learners’ knowledge, a portfolio assessment can include a variety of options. Typically, the designer will be involved in formal progress assessments, such as teachbacks, project-based assessment, or tests and quizzes.

Consider investigating some apps with AI built in such as Bongo or TL;DR which review recordings and provide feedback instantly. These are ideal to assess skills demonstrated in sales calls, meetings, or coaching sessions.

Whatever assessment methods you choose, support the facilitation team by providing detailed instructions, an answer key, and talking points to aid the debrief.

The Role of Skilled Facilitators

Skilled facilitators will constantly be looking for ways to assess their learners so that they can use that data to make adjustments in the moment.

Although designers may be more involved in summative assessment techniques, facilitators can also assess learners in a summative way, particularly through more informal progress assessments such as

• Classroom participation using the feedback tools.
• Quality of the conversation during discussions.
• Interactivity and engagement throughout the lesson. Most virtual classroom platforms offer some kind of analytics on engagement. Be mindful of over-relying on these as some simply indicate the amount of time the classroom was the "active window" on the learner’s computer. If the learner is taking notes or typing into a workbook on another window, they’d still be engaged, even if the analytics suggest otherwise.

When it comes to formative assessment, designers may have included these elements throughout the program materials, but it falls to the facilitator to conduct them and often to add in their own assessments when they feel the learning is not landing the way it could be.

For example, if the learners took a pretest or pre-survey to evaluate their skillset before a class, the facilitator could use that information to group their learners for breakout activities. Sometimes facilitators like to group their learners into engagement groups where either like-minded people are together or those that could learn from each other are together.

Facilitators should be empowered to add additional periodic quizzing techniques to help them assess learner skill building during the live class. Facilitators should have sufficient digital fluency to choose the best tool for the job in the moment, for example:

• Feedback Tools: Using the reactions, such as thumbs up, green checks and red x's as a way to informally poll the group on topics during teaching allows the facilitator to do a quick inventory of knowledge transfer in the moment and make adjustments as needed.
• Polls: Incorporating polls throughout your virtual classroom instruction allows for a bit more of a formal approach to checking on knowledge transfer. Polls allow facilitators to take that information and apply it in the class or as self-directed work for students to get more practice.
• Surveys: Giving Level 1 and 2 assessments through surveys allows learners to provide information to the facilitator about the emotional side of the learning.

There are so many learning apps available that offer fun and interesting ways to quiz your learners. Apps like Kahoot, Mentimeter, or Slido offer lots of different question types, allowing you to assess learners in multiple ways. What’s more, they can be accessed on any device, so if you have a hybrid learning environment where learners are sharing computers, they can individually participate in quizzes through their phones.

With all that technology to choose from and manage, facilitators will want the support of skilled producers.

Analytical Producer Support

Great producers will adopt an analytical mindset by capturing data throughout the live class that could be useful as part of an evaluation.

For example, if the facilitator asks for reactions to check understanding, the producer can capture the responses with a simple screenshot. This can help the producer and facilitator identify any "muddy points" and change how that content is delivered next time.

Another example would occur when an activity runs way over time. The producer can capture notes on what happened as well as timing, freeing the facilitator to figure out how to cover all remaining key learning points in the allotted time. The producer can pass that data on to the design team to review for future programs.

This shift to analytical mindsets doesn’t happen overnight, and it certainly doesn’t just happen. This shift requires a significant effort from the leadership team.

Learning Leaders' Role in Continuous Assessment

As a Virtual Learning Leader, make it your priority to ask questions, encouraging your team to adopt analytical mindsets. Be curious, keep asking for the data, and encourage your team to share that data and their feedback.

To find out just how successful your virtual classroom training is, reach out to InSync Training to learn about our Team Assessment and Coaching Services. Together, with our combined analytical mindsets, we can advance best practices and set new benchmarks for virtual learning