A novel spatio-temporal registration framework for video copy localization based on multimodal features

Fighting movie piracy requires copy detection followed by the accurate frame alignments of master and copy videos, in order to estimate distortion model and capture location in a theater. Existing research on pirate video registration utilizes only visual features for aligning pirate and master videos, while no effort is made to employ acoustic features. Further, most studies in illegal video registration concentrate on the alignment of watermarked videos, while few attempts are made to address the alignment of non-watermarked sequences. We attempt to solve these issues, by proposing a novel spatio-temporal registration framework that utilizes content-based multimodal features for frame alignments. The proposed scheme includes three stages: first, a video sequence is compactly represented using Speeded Up Robust Features (SURF) and audio spectral signatures; second, sliding window based dynamic time warping (DTW) is employed to compute temporal frame alignments; third, robust SURF descriptors are utilized to generate accurate geometric frame alignments. The results of experiments on three different datasets demonstrate the robustness and efficiency of the proposed method against various video transformations.

► We focus on spatio-temporal alignment of pirate videos using multimodal features.
► Study in copy video registration use visual data, no efforts to exploit audio data.
► New spatio-temporal visual fingerprints and audio features for temporal alignments.
► Robust SURF descriptors are utilized to generate geometric frame alignments.
► Results on three different datasets prove robustness and efficiency of proposed method.