A reanalysis of crossed-dimension “Who Said What?” paradigm studies, using a better error base-rate correction

Cognitive and evolutionarily-minded researchers have increasingly adopted the “Who Said What?” memory-confusion paradigm, a powerful and sensitive paradigm originating from social psychology which allows researchers to unobtrusively measure social categorization. The paradigm has been particularly important over the past two decades for arbitrating between different functionalist hypotheses about the evolved social mind. Bor (2018) has pointed out, however, that the simple arithmetic base-rate correction inherited from social psychology for this paradigm is problematic. This standard base-rate correction-in use since 1992 and in over a dozen studies-creates a mathematical artifact in which the calculated magnitude of categorization by one dimension can affect the calculated magnitude of categorization by a second, crossed dimension, even when the two dimensions are in fact fully independent from one another. No one had noticed this in 25 years. Worryingly, this means that all previously-reported “Who Said What?” studies featuring two crossed dimensions have reported potentially-biased estimates of the true magnitude of categorization. Here, a reanalysis of four large research projects is presented, involving 56 effect sizes across 31 between-subjects conditions (Pietraszewski, Cosmides, & Tooby, 2014; Pietraszewski & Schwartz, 2014; Pietraszewski, Curry, Petersen, Cosmides, & Tooby, 2015; Pietraszewski, 2016). In all cases, results from those papers hold or are even strengthened by the reanalysis. In particular, the previously-reported experimental reductions in racial categorization were in fact underestimates. This reanalysis also reveals that the most common artifact of the previously-used error correction was to artificially-inflate categorization by the weaker of the two dimensions-at least among the studies reported here. Finally, a succinct list of best practices for use of the “Who Said What?” paradigm in the future is presented, including but not limited to this new base-rate correction.