Cooperative passers-by tracking with a mobile robot and external cameras

In this paper, we present a cooperative passers-by tracking system between fixed view wall mounted cameras and a mobile robot. The proposed system fuses visual detections from wall mounted cameras and detections from a mobile robot–in a centralized manner–employing a “tracking-by-detection” approach within a Particle Filtering strategy. This tracking information is then used to endow the robot with passers-by avoidance ability to facilitate its navigation in crowds during the execution of a person following mission. The multi-person tracker’s ability to track passers-by near the robot distinctively is demonstrated through qualitative and quantitative off-line experiments. Finally, the designed perceptual modalities are deployed on our robotic platform, controlling its actuators via visual servoing techniques and free space diagrams in the vicinity of the robot, to illustrate the robot’s ability to follow a given target person in human crowded areas.

► We combine fixed view wall mounted cameras and a mobile robot cooperatively for passers-by perception. ► The perception is used to implement a passers-by avoidance modality for a mobile robot. ► Designed perceptual modalities are deployed on our robotic platform. ► The framework is demonstrated with target person following in crowded scenes by the mobile robot.