Home > Topics > Return on Investment >

The Human Capital Return On Investment

A paper from Global Learning Alliance and Knowledge Advisors

Part 2 - Learning Measurement Levels 1-4 (Kirkpatrick)

Knowing there is a definitive need to measure the impacts of a large corporate cost like learning it is fitting to have an industry acceptable model for doing so. This model is actually one that has been in existence since the 1950's but continues to be accepted today using technology and creativity to maximize its benefits for the modern corporation.

In 1959, Donald L. Kirkpatrick, author, PhD, consultant, past president of the ASTD and KnowledgeAdvisors Advisory Board Member published a series of four articles called "Techniques for Evaluating Training Programs." The articles described the four levels of evaluation that he had formulated based on his work for his PhD dissertation at the University of Wisconsin, Madison. Later, Kirkpatrick wrote a book (Donald L. Kirkpatrick, Evaluating Training Programs: The Four Levels, 2nd Edition, Berrett-Koehler Publishers, Inc, San Francisco, 1998) and it is now in its second edition. This book was a source for the information on the following pages related to Levels One through Four.

Kirkpatrick's goal was to clarify what evaluation meant. The model clearly defined evaluation as meaning "measuring changes in behavior that occur as a result of training programs." The model itself is composed of four Levels of training evaluation. A fifth level, ROI has been added since then. The fifth level was the brainchild of Dr. Jack J. Phillips, Ph.D., author, consultant and KnowledgeAdvisors advisory board member and strategic partner. The illustration below and subsequent commentary summarize Kirkpatrick's Four Levels and Phillips' Fifth Level.

Level One - Reaction

Per Kirkpatrick, "evaluating reaction is the same thing as measuring customer satisfaction. If training is going to be effective, it is important that students react favorably to it."

The guidelines for Level One are as follows:

• Determine what you want to find out
• Design a form that will quantify the reactions
• Encourage written comments and suggestions
• Strive for 100% immediate response
• Get honest responses
• Develop acceptable standards
• Measure reactions against standards, and take appropriate action
• Communicate reactions as appropriate

The benefits to conducting Level One Evaluations are:

• A proxy for customer satisfaction
• Immediate and real-time feedback to an investment
• A mechanism to measure and manage learning providers, instructors, courses, locations, and learning methodologies
• A way to control costs and strategically spend your budget dollars
• If done properly, a way to gauge a perceived return on learning investment

Level Two - Learning

Level Two is a 'test' to determine if the learning transfer occurred. Per Kirkpatrick, "It is important to measure learning because no change in behavior can be expected unless one or more of these learning objectives have been accomplished. Measuring learning means determining one or more of the following."

• What knowledge was learned?
• What skills were developed or improved?
• What attitudes were changed?

The Guidelines for Level Two are as follows:

• Use a control group, if practical
• Evaluate knowledge, skills, and or attitudes both before and after the program
• Use a 'test' to measure knowledge and attitudes
• Strive for 100% response
• Use the results to take corrective actions

The benefits to conducting Level Two Evaluations are:

• Learner must demonstrate the learning transfer
• Provides training managers with more conclusive evide nce of training effectiveness

Level Three - Behavior

Level Three evaluates the job impact of training. "What happens when trainees leave the classroom and return to their jobs? How much transfer of knowledge, skill, and attitudes occurs?" Kirkpatrick questions, "In other words, what change in job behavior occurred because people attended a training program?"

The Guidelines for Level Three are as follows:

• Use a control group, if practical
• Allow time for behavior change to take place
• Evaluate both before and after the program if practical
• Survey or interview trainees, supervisors, subordinates and others who observe their behavior
• Strive for 100% response
• Repeat the evaluation at appropriate times

The benefits to conducting Level Three evaluations are as follows:

• An indication of the 'time to job impact'
• An indication of the types of job impacts occurring (cost, quality, time, productivity)

Level Four - Results

Per Kirkpatrick, Level Four is "the most important step and perhaps the most difficult of all." Level Four attempts to look at the business results that accrued because of the training.

The Guidelines for Level Four are as follows:

• Use a control group if practical
• Allow time for results to be achieved
• Measure both before and after the program, if practical
• Repeat the measurement at appropriate time
• Consider costs versus benefits
• Be satisfied with evidence if proof not possible

The advantages to a Level Four evaluation are as follows:

• Determine bottom line impact of training
• Tie business objectives and goals to training

Part 1 < Back to Top > Part 3

© 2004 Global Learning Alliance and Knowledge Advisors. Reproduced with permission. Any opinions or views contained in this article are solely those of the author and do not necessarily represent those of Training Reference.

Books for training professionals

• Bookshop home page
• Training evaluation books
• Training Needs Analysis (TNA) books

Source suppliers

Visit the Training Reference Directory to view supplier details for a wide range of courses, products and services. Related categories for this section include:

• Evaluation services
• Testing/assessment software

Sponsored links

Back to top