Reducing the effect size of the retest effect: Examining different approaches

- Retest form and item administration format were experimentally varied.
- Retest effects differed as a function of experimental conditions.
- Psychometric g only moderated the retest effect in some experimental conditions.
- Learning test-specific skills during test-taking explains retest score gains.
- Computerized adaptive testing reduces the size of retest score gains.

Allowing test-takers to retake cognitive ability tests has been shown to increase test-takers' test scores. Three models differing regarding to the processes assumed to be responsible for this effect have been advanced in the literature. These models make competing predictions (1) regarding the level of measurement invariance across test administration sessions, and (2) regarding the extent to which effect sizes of the retest effect can be minimized by means of alternate retest forms and different item administration formats (fixed-item linear tests vs. computerized adaptive tests) on the effect size of retest effects. The present study systematically varied the item administration format and the retest form across five experimental conditions. Test-takers (N = 960) were randomly assigned to these conditions and solved four cognitive ability tests at two time-points of measurement (test-retest interval ≈ 1 month). Measurement invariance analyses and analyses of the effect sizes of the retest effects across experimental conditions were in line with a model attributing retest score gains to learning during test-taking. The results also indicated that the effect size of the retest effect can be substantially reduced by administering computerized adaptive test forms at the initial test administration session in order to prevent learning during test-taking.