Training effectiveness
Refers to the benefits that the company and the trainees receive from training.
Training outcomes or criteria
Refer to measures that the trainer and the company use to evaluate training programs.
Training Evaluation
Refers to the process of collecting the outcomes needed to determine whether training is effective.
Evaluation Design
Refers to the collection of information-- including what, when, how, and from whom-- that will be used to determine the effectiveness of the training program.
Formative Evaluation
Refers to the evaluation of training that takes place during program design and development. Helps ensure that: the training program is well organized and runs smoothly and trainees learn and are satisfied with the program.
Pilot Testing
Refers to the process of previewing the training program with potential trainees and managers or with other customers.
Summative evaluation
Refers to an evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training program.
The evaluationg process
- 1. Conduct a Needs Analysis
- 2. Develop Measurable Learning Objectives and Analyze Transfer of Training
- 3. Develop Outcome Measures
- 4. Choose and Evaluation Strategy
- 5. Plan and Execute the Evaluation
Kirkpatrick's Four-level Framework of Evaluation Criteria
- 4. Results: Business results achieved by trainees
- 3. Behavior: Improvement of behavior on the job
- 2. Learning: Acquisition of knowledge, skills, attitudes, behavior
- 1. Reactions: Trainee satisfaction
Reaction Outcomes
Refer to trainees perceptions of the program, including the facilities, trainers, and content.
Evaluation Outcomes
- Reactions
- Learning or Cognitive
- Behavior and Skills
- Affective
- Results
- Return on Investment
Cognitive Outcomes
are used to determine the degree to which trainees are familiar with principles, facts, techniques, procedures, or processes emphasized in the training program.
Skill-Based outcomes
Are used to assess the level of technical or motor skills and behaviors.
Affective Outcomes
Include attitudes and motivation.
Results
Are used to determine the training program's payoff for the company
Return on Investment (ROI)
Refers to comparing the training monetary benefits with the cost of the training.
Direct costs
Include: salaries and benefits for all employees involved in training
Indirect costs
Are not related directly to the design, development, or delivery of the training program. General office supplies, facilities, equipment, and related expenses.
Benefits
Are the value that the company gains from the training program
-
Criteria relevance
Refers to the extent to which training outcomes are related to the learned capabilities emphasized in the training program.
Criterion contamination
Refers to the extent that training outcomes measure inappropriate capabilities or are affected by extraneous conditions.
Criterion Deficiency
Refers to the failure to measure training outcomes that were emphasized in the training objectives.
Reliability
Refers to the degree to which outcomes can be measured consistently over time.
Discrimination
Refers to the degree to which trainees' perfomance on the outcome actually reflects true differences in performance.
Practicality
Refers to the ease with which the outcome measures can be collected.
Training Evaluation Practices
Training Program Objectives and Their Implications for Evaluation
Threats to validity
Refer to factors that will lead and evaluator to question either the believability of the study results or the extent to which the evaluation results are generalizable to other groups of trainees and situations
Threats to validity
- Internal Validity: Company, Persons, Outcome Measures.
- External Validity: Reaction to pretest, reaction to evaluation, interactions of selection and training, interaction of methods.
Pretraining measure
Baseline of the outcome.
Posttraining measure
Measure of outcome after training
Comparison group
Refers to a group of employees who participate in the evaluation study but do not attend the training program
Hawthorne Effect
Refers to employees in an evaluation study performing at a high level simply because of the attention they are receiving.
Random Assignment
Refers to assigning employees to the training or comparison group on the basis of chance.
Comparison of Evaluation Designs
- Posttest Only
- Pretest/posttest
- Posttest only with comparison group
- Pretest/posttest with comparison group
- Time series
- Time series with comparison group and reversal
- Solomon Four-group
Factors that influence the type of evaluation design
Change potential, Importance
Scale, Purpose of training, Organization Culture
Expertise, Cost, Time frame
Cost Benefit Analysis
The process of determining the economic benefits of a training program using accounting methods that look at training costs and benefits.
Utility Analysis
Cost benefit analysis method that involves assessing the dollar value of training based on estimates of the difference in job performance betwee trained and untrained employees, the number of individuals trained, the length of time a training program is expected to influence performance, and the variability in job performance in the untrained group of employees.
Success cases
Refer to concrete examples of the impact of training that show how learning has led to results that the company finds worthwhile and the managers find credible.
Training Metrics
Expenditure/employee, learning hours recieved/employee, expenditure as a % of payroll, expenditure as a % of revenue, cost/learning hour received, % of expenditures for external services, Learning hours received/training and development staff member, average % of learning activities outsourced, average % of learning contant by content area, Average % of learning hours provided via different delivery methods.
Refers to the evaluation of training that takes place during program design and Development
Training evaluation is the systematic process of collecting information and using that information to improve your training. Evaluation provides feedback to help you identify if your training achieved your intended outcomes, and helps you make decisions about future trainings.
trainees' perceptions of the program, including the facilities, trainers, and content (Reaction outcomes are often referred to as a measure of "creature comfort."
Training evaluation models are methods used to evaluate training programs. The 5 most popular are Kirkpatrick's four-level training evaluation model, The Phillips ROI model, Kaufman's five levels of evaluation, The CIRCO model, and Anderson's model of learning evaluation.
Following are the methods of training evaluation:.
Satisfaction and Participant reaction. ... .
Knowledge Acquisition. ... .
Measuring the Business Improvement. ... .
Return on Investment (ROI) ... .
Describing the outputs. ... .
Pre-training Assessment. ... .
Post-Assessment (reactions) ... .
Post-Assessment (learning).