Scene-adaptive video partitioning by semantic object tracking

An adaptive mechanism for video partitioning by semantic objects tracking is proposed. A video scene consists of the sequence of frames between two adjacent video scene changes which can be detected according to the video scene complexity. In general, the video scene complexity can be described in twofold characteristics—the temporal domain motion complexity and the spatial domain activity complexity. For this purpose, we propose a novel spatial-temporal segmentation method as a general segmentation algorithm combining several types of information including color and motion. A region within a foreground object is called as a foreground region, which is characterized as a moving uniform region. An algorithm for object tracking based on the foreground regions is also included in order to recognize camera and object movements and obtain correct video shots. By analyzing foreground objects between consecutive frames, the types of scene change and the types of camera movement can be detected according to the number of entering and existing regions and the motion vectors, respectively. Based on these parameters, the frames of a video sequence are categorized into normal, cut, fade, and dissolve classes. Adaptation is realized by grouping variable number of the labeled frames as a unit, which contains a scene change to be automatically determined by the moment-preserving thresholding techniques. Experimental results are presented to demonstrate the performance of the new method in terms of better segmentation.