Goal-oriented top-down probabilistic visual attention model for recognition of manipulated objects in egocentric videos

• We propose a new top down probabilistic saliency model for egocentric video content.
• Top-down saliency maps focused on manipulated objects based on the arms positions.
• Psycho-visual experiment conducted demonstrating our proposal increased performances over SOA saliency models.
• SOA performances in manipulated object recognition for this dataset.

We propose a new top down probabilistic saliency model for egocentric video content. It aims to predict top-down visual attention maps focused on manipulated objects, that are then used for psycho-visual weighting of features in the problem of manipulated object recognition. The model is probabilistically defined using both global and local appearance features extracted from automatically segmented arm areas and objects. A psycho-visual experiment has been conducted in a guided framework that compares our proposal and other popular state-of-the-art models with respect to human gaze fixations. The obtained results show that our approach outperforms several popular bottom-up saliency approaches in a well-known egocentric dataset. Furthermore, an additional task-driven assessment for object recognition in egocentric video reveals that the proposed method improves the performance of several state-of-the-art techniques for object detection.