Neural-swarm visual saliency for path following

This paper extends an existing saliency-based model for path detection and tracking so that the appearance of the path being followed can be learned and used to bias the saliency computation process. The goal is to reduce ambiguities in the presence of strong distractors. In both original and extended path detectors, neural and swarm models are layered in order to attain a hybrid solution. With generalisation to other tasks in mind, these detectors are presented as instances of a generic neural-swarm layered architecture for visual saliency computation. The architecture considers a swarm-based substrate for the extraction of high-level perceptual representations, given the low-level perceptual representations extracted by a neural-based substrate. The goal of this division of labour is to ensure parallelism across the vision system while maintaining scalability and tractability. The proposed model is shown to exhibit, at 20 Hz, a 98.67% success rate on a diverse data-set composed of 39 videos encompassing a total of 29,789 640 × 480 frames. An open source implementation of the model, fully encapsulated as a node of the Robotics Operating System (ROS), is available for download.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slideHighlights► We report on a neural-swarm layered architecture for visual saliency computation. ► We validate the neural-swarm architecture on vision-based robot path following. ► Neural layer useful for the modelling of highly dense isotropic local connectivity. ► Swarm layer useful for the integration of spatiotemporal anisotropic local contexts. ► The model shows a success rate of 98.67% at 20 Hz on man-made and natural paths.