Detecting when timeseries differ: Using the Bootstrapped Differences of Timeseries (BDOTS) to analyze Visual World Paradigm data (and more)

In the last decades, major advances in the language sciences have been built on real-time measures of language and cognitive processing, measures like mouse-tracking, event related potentials and eye-tracking in the visual world paradigm. These measures yield densely sampled timeseries that can be highly revealing of the dynamics of cognitive processing. However, despite these methodological advances, existing statistical approaches for timeseries analyses have often lagged behind. Here, we present a new statistical approach, the Bootstrapped Differences of Timeseries (BDOTS), that can estimate the precise timewindow at which two timeseries differ. BDOTS makes minimal assumptions about the error distribution, uses a custom family-wise error correction, and can flexibly be adapted to a variety of applications. This manuscript presents the theoretical basis of this approach, describes implementational issues (in the associated R package), and illustrates this technique with an analysis of an existing dataset. Pitfalls and hazards are also discussed, along with suggestions for reporting in the literature.