The neural architecture of prediction over a continuum of spatiotemporal scales

Theories of time and space in memory have traditionally focused on their role in dividing experience into discrete episodes, despite the arbitrary nature of these divisions. We offer an alternative characterization that focuses on the fundamentally predictive role of perception and memory. In this account, perceptual hierarchies in sensory cortex detect patterns of feature-change across a logarithmic continuum of scales in time and space, which allows them to efficiently converge on nuanced, yet short-range, predictions of the present situation. Time and space emerge from this continuum as representations of feature-distance that provide a measure of the relevance of non-simultaneous experiences, allowing for long-range associations, mental time-travel, and predictions that go far beyond the immediate moment. This reframing of the nature and role of time and space in memory has implications for both the interpretation of existing findings and the design of future experiments.