A distributed perception infrastructure for robot assisted living

•We developed a novel distributed infrastructure based on open-source middleware.•We realized an ambient intelligence system for assisted living.•The system does not require people cooperation.•Cooperation of the audio and video sensors increases performance and reliability.•Integration with a robot for performing actions on the environment.

This paper presents an ambient intelligence system designed for assisted living. The system processes the audio and video data acquired from multiple sensors spread in the environment to automatically detect dangerous events and generate automatic warning messages. The paper presents the distributed perception infrastructure that has been implemented by means of an open-source software middleware called NMM. Different processing nodes have been developed which can cooperate to extract high level information about the environment. Examples of implemented nodes running algorithms for people detection or face recognition are presented. Experiments on novel algorithms for people fall detection and sound classification and localization are discussed. Eventually, we present successful experiments in two test bed scenarios.