MVS: A multi-view video synopsis framework

•A framework to generate one synopsis of multi-view videos is proposed.•The synopsis is visualized by mapping each camera view to a common ground plane.•The generated synopsis also incorporates the advantages of video summarization.•The action recognition module helps limiting the synopsis length significantly.•Visibility scores of each object track further reduce the synopsis length.

In this paper, we present a framework for generating a synopsis of multi-view videos that are acquired from a surveillance site, indoor or outdoor, using multiple cameras. The synopsis generation is modeled as a scheduling problem that we solve using three separate approaches: table-driven approach, contradictory binary graph coloring (CBGC) approach, and simulated annealing (SA) based approach. An action recognition module is included in the framework to recognize important actions performed by various humans present in the videos. Inclusion of such important actions in the synopsis has helped to reduce its length significantly. The synopsis length is further reduced through a post-processing step that computes the visibility score for each object track using a fuzzy inference system. Among the three proposed schemes, maximum reduction in synopsis length is obtained through the CBGC approach. The stochastic approach using SA, on the other hand, achieves a better trade-off among the multiple optimization criteria. Experimental evaluations on standard datasets demonstrate the efficacy of the proposed framework over its counterparts concerning the reduction in synopsis length and retention of important actions.