Developing an Evaluation Plan that Works

Maintaining exceptional quality in the face of rapid growth is a common challenge faced by many organizations that find early success. Professional development is often seen as an answer to standardization and quality assurance issues among geographically disparate or rapidly growing teams. It's a great solution, but it always gives rise to another issue:

How are we supposed to know when the project is successful? Though it is usually an afterthought, program evaluation is an integral component of every project. As simple as it can be, it is also easy to deprioritize. Luckily, assessment of learning outcomes is one of my favorite instructional design activities. 

Begin with the end in mind

Contrary to popular belief, my work actually starts at the beginning by ensuring a solid foundation for a comprehensive evaluation. Well designed goals and objectives are the cornerstone of an effective evaluation, which is why I harp so much on goals and objectives provide technical assistance in goal analysis and performance objective design. From there I can develop evaluation instruments tailored to the objectives and aesthetics of the course.

Improvement planning

Systematic data collection and analysis is a must to ensure the integrity of your findings. Methodologies corresponding to each level of evaluation are listed in the table below in conjunction with measurable indicators of success.

The final stage of evaluation is critical, and yet most often forgotten. The primary purpose of evaluation is program improvement; I assist in identifying and addressing target improvement areas to ensure continued success. 

If you don't have K&K's book. Buy it. Trust me.

Easy as 1, 2, 3...4

I'm referring to Kirkpatrick and Kirkpatrick's (2006) model, of course. Those guys really make it easy. Using their incredibly practical methodology I outline evaluation tools that gauge the course's success on four levels:

1. Reaction

2. Learning 

3. Behavior 

4. Results 

The levels work like gears, each one essential to the functioning of the other and each one contributing to the overall operation of the course. 

Here's a sample four-level intervention strategy

The information gleaned from a well-designed evaluation can promote credibility and validation of the training materials and methods employed.  And the benefits of evaluation do not end there. Evaluations provide

• Quantifiable feedback that highlights improvable areas in your program. You’ll have clear targets for refining instruction the next time around. 

• A robust understanding of your program. Familiarity with every nook and cranny means no surprises down the road! 

• The opportunity to show the effects of monetary and human capital investments on your organization's bottom line illustrated through cost/benefit analytics and ROI calculation.

Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006).  Evaluating Training Programs.  Berrett-Koehler Publishers, Inc.:  San Francisco.