Introducing new material for a standardized exam: a controlled, prospective, double-blinded test of student learning

Do multiple choice unit tests reflect what students have learned during the unit? The day before the administration of a county-mandated multiple choice test, two classes were shown a topic they had not previously seen, and told it would be on the test. One class was shown the same material and told it was not important, and two classes were not shown the material. There was one question on the test concerning this material. The first two classes (n=54) had 83% correct on the question. The next class (n=32) had 81% correct. The last two classes (n=43), who did not see the material, had 63% correct on the question. The results were statistically significant, p=0.045. There was no difference between the class averages on the test overall. The results appear to indicate that one can prepare for multiple choice tests independently of other aspects of learning.