Training Evaluation PDF

Training Evaluation Evaluation is not at the End Ae De De Ie E Kirkpatrick Four-Level Framework Level 4: Results (Outcome of...

Training Evaluation Evaluation is not at the End Ae De De Ie E Kirkpatrick Four-Level Framework Level 4: Results (Outcome of application) Level 3: Behavior (Application) Level 2: Learning (Learning KSA) Level 1: Reaction (Satisfaction with the Program) The New World Kirkpatrick Framework http://www.trainingindustry.com/media/16099044/new_world_kirkpatrick_model.jpg  Philips ROI Process Model Data Analysis Tabulate Program Planning Data Collection Costs Reporting Develop Evaluation Reaction/ Business Convert Data Return on Learning Application Plans and Satisfaction Impact To Monetary Investment Reach Level 2 Level 3 Base Line Level 1 Level 4 Value Level 5 Conclusion And Generate Data Report Communicate Information Identify Isolate To Target Intangible Effects of Groups Benefits Program  Chain of Impact React to program Obtain skills/knowledge Apply skills/knowledge Change in Business measures Isolate the Effects Generate ROI When higher levels of evaluation is conducted, data must be collected at lower levels  Control group Trend lines Methods of Participants' estimates Isolation Participants’ supervisors’ estimates Experts estimates  After collecting performance data, divide them into hard data and soft data Converting Hard data Output, Time, costs, Data to and quality Monetary Work habits, customer service, job Benefit Soft data: attitudes, and initiatives Convert the data to monetary values  Convert output data to profit contribution or cost savings Calculate cost of quality & convert quality improvements to cost savings Methods Use the trainees’ CTC as value of time Use internal and external experts Use estimates from external data base  Cost Items Prorated Expensed 1. Initial Analysis & assessment * 2. Program design & * development 3. Program implementation Cost Items a. L&D team time * b. Coordination time * c. Participants’ time * d. Program material * e. Hardware/software * f. Travel/lodging/meals *  Cost Items Prorated Expensed g. Use of facilities * 4.Maintanince & * Cost Items monitoring (Cont.) 5. Administrative * support & overhead 6. Evaluation & * reporting  CBR= Program benefits divided by program cost Calculate ROI ROI ( %)= Net program benefit multiplied by 100 and divided by program cost  Evaluating ROI is complicated and time consuming The cost element of the training is easy to arrive at Complicated The benefit values are subjective and ROI difficult to quantify Organizations should implement ROI impact studies selectively Select one or two programs for initial calculation  Brinkerhoff’s Success Case Method Document business impact to disseminate Conduct within company Survey of evaluation Participants studies with few successful & unsuccessful Document participants factors that enhance or impede business impact Comparison of Training Evaluation Models Kirkpatrick's Four Kirkpatrick’s Return Phillips’s ROI Brinkerhoff’s Success Levels on Expectations Process Model Case Method Level 1:Reaction The four levels in Level 1: Reactions Focus on the trainees reverse - The end is who are successful and the beginning who are not successful & why? Level 2: Learning ROE is the ultimate Level 2: Learning Step 1: Map the program indicator of value with business needs Level 3: Behavior Business Level 3: Behavior Step 2: Survey partnership Level 4: Results Chain of evidence Level 4: Results Step 3: gather data Level 5:ROI Step 4: analyze data Step 5: Communicate the findings  Comparison of Training Evaluation Models (Cont.) Kaufman Five CIRO Model Hamblin's Five Levels CIPP Model Levels Model (Warr, Bird, & of Evaluation (Galvin) Rackham) 1 a. Enabling Context 1. Reactions Context 1 b. Reactions 2. Acquisition Input 2. Learning Input 3. Application Reactions 3. Behavior Process 4. Organizational Outcome 4. Functioning (cost- Product output benefit) 5. Societal 5. Ultimate (survival & outcomes profitability)