Learning on the Establishment of Training Effectiveness Evaluation and Tracking System
On the 39th Meeting of the 11st Session of the Examination Yuan held on June 18, 2009, the medium-range program “Establishment of Training Effectiveness and Tracking System" of the fourth program “Strengthening the Training System and Enhancing Senior Civil Service” of the “Civil Service System Reform Plan” was approved, for the purpose of strengthening the evaluation and assessment during the training period, establishing the post-training evaluation mechanism, following-up the training effectiveness and deliberating the evaluation of behavior level, and results levels, in favor of the full use of training result in the working process, integrate the training and use as a whole, and to use as a reference in improving the training system.
In 2000, the Civil Service Protection and Training Commission (CSPTC) appointed Nien-Chi Liu, Associate Professor of the National Central University of Human Resource Management Department to conduct the “ Research on the Establishment of Training Effectiveness Evaluation and Tracking System” from June 01,2010 to December 20, 2010. Results obtained from the studies on the fundamental training for examination qualifiers, the training for rank promotion and training for senior civil servantsheld by the CSPTC, and recommendations with respect to such studies, are summarized as follows:
1. Results obtained from the evaluation of three trainings of the CSPTC:
（1） Currently, the conditions for effectiveness evaluation learning on the first and the second levels(reaction level and learning levels) of trainings have been satisfactory, and discussions and modifications should be conducted for only a part of the training content. However, with respect to the after-class tracking and evaluation for the third behavior levelsof training, there is lack of a coping mechanism.
（2） Although criteria of current training reaction levels have reached levels, there is still space for their improvement.
（3） Most of the trainees completing the fundamental training for senior examinations agree that the training objective set for the training can be reached with the training courses, in which courses of laws concerning civil service and administrative management knowledge and skill are most helpful. Trainees participatingJunior to senior Rank Promotion Training also agree that the training objective set for the training can be reached with the training courses, but they also bring forward their comments on the overall design for training, after-class assessment method, the training schedule and the training method.
（4） In the current civil service training system, behavior levels and results levels, level criteria are seldom used in training evaluation. In particular, in the aspect of results level criteria, there is no information of actual implementation in the US, Canada and Singapore. However, it does not mean that the evaluation criteria of these two levels of training are not applicable to the civil service training system of the R.O.C. At present, there is an urgency to identify the role of the evaluation criteria of these two levels of training in the overall training mode.
2. Recommendations with respect to evaluation mechanism for three trainings of the CSPTC:
（1） Modifications for evaluation mechanism for three trainings of the CSPTC are summarized as follows:
1. Training for examination qualifiers and rank promotion:
With respect to reaction levels and learning levels, it is recommended that questionnaires for courses and programs be supplemented, which should be issued immediately after the completion of courses and programs. In regards to behavior levels and results levels, it is recommended that a behavior tracking survey form be designed as the evaluation tool for behavior plan, and be reviewed within three to six months upon the completion of training.
2. Mid-long term training for senior civil servants:
With respect to reaction levels and learning levels, it is recommended that questionnaires for courses and programs be supplemented, which should be issued immediately after the completion of courses and programs. In regards to relevant behavior levels and result levels, it is recommended that a behavior tracking survey form be designed as the evaluation tool for behavior plan along with a review mode based on core competencies be established, and be reviewed within three to six months upon the completion of training. At the same time, a capability review model may be set up as the foundation for future surveys for review within six to twelve months upon the completion of training.
（2） Recommendations with respect to the implementation of training evaluation are as follows；
1. The evaluation criteria for reaction levels should be divided into courses and programs.
2. The evaluation criteria for learning levels shall be selected in an appropriate manner in light of the content of courses.
3. The tracking survey result of training effectiveness in the evaluation of behavior levels should be used for modification of training courses, or as the information source required in planning future training.
4. In the future tracking evaluation on training in behavior levels, matters related to transfer of training may be further included, to understand the environmental support of trainees in using knowledge and skills obtained in the training process in their work.
The CSPTC will, in consideration of the aforesaid conclusions and recommendations, deliberate the amendment (2011), for the purpose of modifying questionnaires for the trainings and reaction levels, reviewing the evaluation method for learning levels, planning the evaluation on future promotion activities, and gradually establishing the tracking system.