A comparative analysis of common mistakes in achievement tests prepared by school teachers and corporate trainers

Test development has been an important part of measurement and evaluation in any educational setting, whether its purpose is instruction or training. Both teachers and trainers are expected to have certain level of mastery in developing reliable and valid tests for assessing performance of learners adequately. However, it has often been reported that teachers and trainers are not competent enough to develop such tests. Most teachers have taken at least one university course on educational measurement and evaluation during their undergraduate education. Similarly, a considerable number of corporate trainers attend at least one in-service training seminar on measurement and evaluation as a part of their professional training program. However, teachers and trainers still make serious mistakes in preparing tests for assessing the level of learning and performance properly. This study compares their mistakes based on basic principles of test development. More specifically, the researcher wanted to assess whether the mistakes of teachers and trainers are similar or different. Toward this purpose, a total of 120 instructors (62 teachers and 58 trainers) were selected as participants of the study. A total of 6450 test items in various fields of learning were analysed to make comparisons. Results generally suggest that school teachers and corporate trainers make similar mistakes, although their level of knowledge and skills in measurement and evaluation are different due to prior learning. There is also no difference regarding the subject matter area and the level of education for which the tests were developed. Approximately 60% of items need revisions, half of which appear to have severe mistakes. The most common problems are related to easy test items, route learning, implausible distractors, use of negative questions, hidden cues, illogical order of alternatives, length of sentences for correct answers, disturbing sequence of items, single-mode (text-based) questions, and subjective items. These results, along with some others, have serious implications for programs on teacher education and training of trainers.