A spatio-temporal Long-term Memory approach for visual place recognition in mobile robotic navigation

• We propose a novel biologically inspired neural network architecture for the problem of visual topological indoor localization.
• KK-Iteration Fast Learning Neural Network and Spatio-temporal Long-Term Memory structure were combined to process image sequences.
• Competitive results were obtained on the challenging COLD dataset.

This paper proposes a solution to the problem of mobile robotic localization using visual indoor image sequences with a biologically inspired spatio-temporal neural network approach. The system contains three major subsystems: a feature extraction module, a scene quantization module and a spatio-temporal long-term memory (LTM) module. During learning, the scene quantization module clusters the visual images set into scene tokens. A KK-Iteration Fast Learning Artificial Neural Network (KFLANN) is employed as the core unit of the quantization module. The KFLANN network is driven by intrinsic statistics of the data stream and therefore does not require the number of clusters to be predefined. In addition, the KFLANN performance is less sensitive to data presentation ordering compared to popular clustering methods such as kk-means, and can therefore produce a consistent number of stable centroids. Using scene tokens, the topological structure of the environment can be composed into sequences of tokens. These sequences are then learnt and stored in memory units in an LTM architecture, which is able to continuously and robustly recognize the visual input stream. The design of memory units addresses two critical problems in spatio-temporal learning, namely error tolerance and memory forgetting. The primary objective of this work is to explore the synergy between the strength of KFLANN and LTM models to address the visual topological localization problem. We demonstrate the efficiency and efficacy of the proposed framework on the challenging COsy Localization Dataset.