How to measure scientific reasoning in primary school: A comparison of different test modalities

Investigating scientific reasoning comprehensively with large sample sizes is a challenge, especially in young children where there is little evidence showing that paper-and-pencil instruments can be used to reliably measure competencies. The present study with 122 third graders compares three test modalities: an established interview (Science-K Interview), a novel group test (Science-K Group Test), and an established paper-and-pencil test (Science-P Reasoning Inventory), asking how scientific reasoning can be reliably measured in primary school. The children were presented with 15 items from the Science-K Interview, 15 parallel items from the Science-K Group Test, and 5 items from the Science-P Reasoning Inventory. The interview was presented first, followed by the group test and Science-P items. The Rasch model revealed a good fit of 23 items to the unidimensional model indicating that there is a single underlying ability involved in primary-school scientific reasoning that can be measured with both interview and group test. Although students performed significantly better in the group test than in the interview, the difference was not substantial and might partially be explained by the sequence of the tests. As expected, children solved more Science-K items correctly compared to the Science-P items, which assesses more advanced aspects of scientific reasoning. The important finding of the study is that group tests can reliably capture scientific reasoning in third graders. The use of group tests in primary school can facilitate conducting studies with large sample sizes and can be used for diagnostic purposes, especially in the context of science education in schools.