Data quality over data quantity in computational cognitive neuroscience

We analyzed factors that may hamper the advancement of computational cognitive neuroscience (CCN). These factors include a particular statistical mindset, which paves the way for the dominance of statistical power theory and a preoccupation with statistical replicability in the behavioral and neural sciences. Exclusive statistical concerns about sampling error occur at the cost of an inadequate representation of the problem of measurement error. We contrasted the manipulation of data quantity (sampling error, by varying the number of subjects) against the manipulation of data quality (measurement error, by varying the number of data per subject) in a simulated Bayesian model identifiability study. The results were clear-cut in showing that - across all levels of signal-to-noise ratios - varying the number of subjects was completely inconsequential, whereas the number of data per subject exerted massive effects on model identifiability. These results emphasize data quality over data quantity, and they call for the integration of statistics and measurement theory.