Supporting divergent and convergent production of test items for teachers in higher education

• We examine the effect of an intervention to improve the production of test items.
• The intervention aims to improve the originality and technical quality of test items.
• The intervention improves the degree of originality of test items produced.
• The intervention does not improve the technical quality of test items produced.
• Producing more test items correlates with a lower degree of technical quality.

Developing tests items for end-of-course tests in higher education is a recurring task for many teachers. Divergent production of test items is needed to create queries with original content, while convergent production is needed to design items that are clear and technically sound. Support for the test item design process, based on divergent and convergent production, could lead to better test items. An experiment to establish the extent to which the explicit application of diverging and converging support resulted in more and in more original test items and technically better test items was conducted in this study. The results showed an effect on the originality of the test items developed, but not on the technical quality of those items. The implications of these findings for test item design are discussed.