Responding to the severe limitations of cross-sectional surveys: Commenting on Rong and Wilkinson’s perspectives

While a meta-analysis is necessary to test the claim that the logic dominates the majority of studies, most studies by academic scholars on thinking and actions by executives appear to rely on cross-sectional surveys that use self-reports by executives via scaled (e.g., strongly disagree to strongly agree) instruments whereby one executive per firm completes the instrument and data are collected for 50–500 firms. Useable response rates in these studies are almost always below 30% of the distributions of the surveys. While these studies are sometimes worthwhile for learning how respondents assess concepts and relationships among concepts, Rong and Wilkinson’s perspective on the severe limits to the value of such studies rings true: such surveys reveal more about executives’ sensemaking processes than the actual processes. The limitations of using one-shot, one-person-per-firm, self-reports as valid indicators of causal relationships of actual processes are so severe that academics should do more than think twice before using such surveys as the main method for collecting data – if scholars seek to understand and describe actual processes additional methods are necessary for data collection. The relevant literature includes several gems of exceptionally high quality, validity, and usefulness in the study of actual processes; identifying these studies is a useful step toward reducing the reliance on one-shot self-report surveys.

► The research method influences theory creation, testing of theory, and interpretation of findings. ► Folk theory of mind helps to understand how executives explain their thinking and behavior. ► Get help finding research gems on thinking, doing, and outcomes of executive behavior-- such gems are rare but very valuable.