Gender differences in figural matrices: The moderating role of item design features

There is a heated debate on whether observed gender differences in some figural matrices in adults can be attributed to gender differences in inductive reasoning/Gf or differential item functioning and/or test bias. Based on previous studies we hypothesized that three specific item design features moderate the effect size of the gender difference by introducing differential item functioning and/or test bias. In a series of three studies (Nstudy 1 = 620; Nstudy 2 = 563; Nstudy 3 = 597) we experimentally manipulated three item design features by means of automatic item generation that have been hypothesized to induce differential item functioning and/or test bias in favor of adult males. Item response theory was used to evaluate differential item functioning, while multi-group mean and covariance analyses enabled the separation of variance components due to the latent inductive reasoning/Gf trait from test bias variance components. The results indicated that more complex figural matrices elements (study 1) and rule direction ambiguity (study 3) constitute item design features, which increase the likelihood of differential item functioning and/or test bias in favor of adult males. By contrast, the use of overlays and fused geometrical elements merely led to a slight decrease of the latent inductive reasoning/Gf saturation with no effect on the likelihood of differential item functioning and/or test bias (study 3). Most interestingly, test bias was most pronounced for the Raven Advanced Progressive Matrices in all three studies; indicating that additional item design features may exist, which induce measurement- and/or test bias in favor of adult males.

► The effect of item design features on the gender difference in matrices tests was evaluated.
► Automatic item generation was used to manipulate three relevant item design features.
► DIF/test bias was separated from Gf using IRT- and multi-group confirmatory factor analyses.
► Two item design features induced DIF/test bias and increased the gender difference.
► The third item design feature only affected the test's Gf.