Video summarization via minimum sparse reconstruction

• A minimum sparse reconstruction (MSR) based video summarization (VS) model is constructed.
• An L0 norm based constraint is imposed to ensure real sparsity.
• Two efficient and effective MSR based VS algorithms are proposed for off-line and on-line applications, respectively.
• A scalable strategy is designed to provide flexibility for practical applications.

The rapid growth of video data demands both effective and efficient video summarization methods so that users are empowered to quickly browse and comprehend a large amount of video content. In this paper, we formulate the video summarization task with a novel minimum sparse reconstruction (MSR) problem. That is, the original video sequence can be best reconstructed with as few selected keyframes as possible. Different from the recently proposed convex relaxation based sparse dictionary selection method, our proposed method utilizes the true sparse constraint L0 norm, instead of the relaxed constraint L2,1L2,1 norm, such that keyframes are directly selected as a sparse dictionary that can well reconstruct all the video frames. An on-line version is further developed owing to the real-time efficiency of the proposed MSR principle. In addition, a percentage of reconstruction (POR) criterion is proposed to intuitively guide users in obtaining a summary with an appropriate length. Experimental results on two benchmark datasets with various types of videos demonstrate that the proposed methods outperform the state of the art.