Evaluating dimensional distinctness with correlated-factor models: Limitations and suggestions

•The correlated factor model preferred by differential cognitive psychologists is ambiguous.•A higher-order model can verify the dimensions implied by a correlated factor model.•Several newly proposed constructs by differential cognitive psychologists may not be valid.

Many differential cognitive psychologists appear to interpret correlated-factor models associated with inter-latent variable correlations meaningfully < 1.0 as support for the plausibility of several related, but to some degree distinct, dimensions. It is contended in this paper that such a conclusion drawn from a well-fitting correlated-factor model may not be justified necessarily, even if the correlated-factor model fits better than a single-factor model. Based on a series of simulated correlation matrices, it is demonstrated that a well-fitting correlated-factor model with inter-latent variable correlations < 1.0 only suggests the possibility of several distinct group-level dimensions. In order to obtain additional useful information relevant to the distinctness of the hypothesized group-level factors, a higher-order model is demonstrated to be particularly useful, especially when complemented with omega hierarchical subscale (omegaHS), an effect size index of unique latent variable strength. The following guidelines are provided to help interpret the magnitude of omegaHS values: relatively small < 0.20; typical 0.20 to 0.30; and relatively large > 0.30. The implications of the simulation are demonstrated based on the re-analysis of three previously published correlations relevant to cognitive processes. Researchers are encouraged to supplement correlated-factor model analyses with higher-order model analyses, in order to evaluate the distinctness of the hypothesized dimensions more fully.