A software framework to integrate vision and reasoning components for Cognitive Vision Systems

Cognitive Vision Systems (CVSys) are systems that combine computer vision with reasoning and semantic interpretation components to achieve high-level tasks. Examples are the interpretation of activities a human executes when handling objects or fetching objects with a companion robot. This application requires the use of a large number of functionalities, e.g., perception-action mapping, recognition and categorisation, prediction, reaction to actions, symbolic interpretation, and communication to humans or other systems. Within this contribution, these cognitive vision functionalities are encapsulated in components. The objective is to provide clearly specified components which can be reused in other cognitive vision or robotics systems. To arrive at the level of building a system from these functionalities, it is considered essential to provide a framework that coordinates the components. The framework is built on the service principle, which uses a “Yellow Pages” directory to announce its capabilities and to select other components. The paper summarises the experiences of integrating the components in a context-oriented system for activity interpretation based on task-driven processing of the components. The discussion highlights the advances in vision with respect to exploitation in autonomous robotic systems.