Using automatic item generation to meet the increasing item demands of high-stakes educational and occupational assessment

The use of new test administration technologies such as computerized adaptive testing in high-stakes educational and occupational assessments demands large item pools. Classic item construction processes and previous approaches to automatic item generation faced the problems of a considerable loss of items after the item calibration phase. In this paper a two-component approach to automatic item generation—the automatic min–max approach—is suggested to resolve parts of this problem during the item construction process itself. The procedural framework of this approach is outlined using number series problems as an example. Two studies were conducted to evaluate the psychometric quality of the automatically generated items using explanatory and descriptive item response theory models. The results indicated that the two-component item construction process has been successful in generation an item pool of construct-valid number series items with known psychometric characteristics with no item loss after the calibration phase.

► Meeting the increased demands of high quality fluid intelligence items.
► Combining cognitive theory and item response theory in test construction.
► Using two-component automatic item generation to minimize item loss.
► Enhancing fairness and construct validity through automatic item generation.