A self-reference problem in test score normalization

It is considered standard practice to transform IRT-scaled test scores into standard normal variables for regression analysis in order to enable comparison with other research whose test scores are similarly transformed. This paper calls this practice into question. I show that these transformations can potentially result in radically different estimates of regression parameters due to differences in sample composition. Regression coefficient comparisons between different samples that use z-standardized test scores is only possible if the samples are considered to be random draws from the same population. I outline several different methods to deal with this problem and the caveats attached to each.