Spatiotemporal saliency detection and salient region determination for H.264 videos

• Spatiotemporal saliency detection and salient region determination are proposed.
• Spatial saliency maps are generated by the phase spectrum of Fourier transform (PFT).
• Temporal saliency maps are generated by the backward accumulated MV fields and PFT.
• The final saliency map is adaptively fused by spatial and temporal saliency maps.

In this study, a spatiotemporal saliency detection and salient region determination approach for H.264 videos is proposed. After Gaussian filtering in Lab color space, the phase spectrum of Fourier transform is used to generate the spatial saliency map of each video frame. On the other hand, the motion vector fields from each H.264 compressed video bitstream are backward accumulated. After normalization and global motion compensation, the phase spectrum of Fourier transform for the moving parts is used to generate the temporal saliency map of each video frame. Then, the spatial and temporal saliency maps of each video frame are combined to obtain its spatiotemporal saliency map using adaptive fusion. Finally, a modified salient region determination scheme is used to determine salient regions (SRs) of each video frame. Based on the experimental results obtained in this study, the performance of the proposed approach is better than those of two comparison approaches.