Comparison of methods for factor extraction for cognitive test-like data: Which overfactor, which underfactor?

Research published in Intelligence showed that the number of factors measured by individual intelligence tests has increased dramatically over time (Frazier & Youngstrom, 2007). When “gold standard” methods (parallel analysis based on principal components analysis, PA-PCA, and minimum average partial, MAP) were applied to these same tests fewer factors emerged, leading the authors to conclude that tests were inappropriately overfactored, and that modern tests are measuring far fewer underlying constructs than they are intended to measure. The article was influential, with the findings cited commonly and used to guide subsequent analyses. Here, we tested a key assumption of Frazier and Youngstrom and subsequent research that has used these methods: whether MAP and PA-PCA are accurate in recovering the correct number of factors with cognitive-test-like data. MAP and PA-PCA were compared to other exploratory and confirmatory factor analytic techniques in their ability to recover the correct number of factors in simulated data. Data conformed to common values found in intelligence literature, and varied based on the number of tests per factor, magnitude of factor loadings and factor correlations, and sample size. Results showed that MAP and PA-PCA, in fact, underfactored under many realistic data conditions, meaning that they recovered too few factors. Confirmatory methods were more accurate. Among exploratory methods PA based on principal axis factoring (not principal components analysis) was most accurate, although all methods underfactored with few tests per factor and high factor correlations. These findings suggest that PA-PCA and MAP are not “gold standard” methods for determining the number of factors underlying intelligence data and that other methods are more accurate. We argue for the importance of formal and informal theory in factor analytic investigations of intelligence tests.