Real-time generation of personalized home video summaries on mobile devices

With the proliferation of mobile devices and multimedia, videos have become an indispensable part of life-logs for personal experiences. In this paper, we present a real-time and interactive mobile application for home video summarization on mobile devices. The main challenge of this method is lack of information about the video content in the following frames, which we term “partial-context” in this paper. First of all, real-time segmentation algorithm based on partial-context is applied to decompose the captured video into segments in line with the change in dominant camera motion. Secondly, an original key frame update strategy is presented to optimize selected key frames in such partial-context. In addition, the main challenge to conventional video summarization is the semantic understanding of the video content. Thus, we leverage the fact that it is easy to get user input on a mobile device and attack this problem through the user interaction. The user preference is learned and modeled by a Gaussian Mixture Model (GMM) whose parameters are updated each time when users manually select a key frame. Our system utilizes the user preferences to optimize the key frame update process. Evaluation results demonstrate that our system significantly improves users experience and provides an efficient automatic/semi-automatic video summarization solution for mobile users.