Mining movies for song sequences with video based music genre identification system

Musical sequences with actors dancing and lip-synching to songs sung by playback singers are integral parts, particularly of South Asian movies. Fans seek out movies for their songs and they often seek songs of a particular genre. In fact, song and dance sequence of South Asian movies are an industry of their own. Given the huge numbers of movies produced in South Asia over the past decades, most of which are in digital archives, it is an important problem to automatically extract and categorise their musical sequences. This paper proposes a system for musical sequences extraction from movies. Our method invokes an SVM-based classifier and makes as well a novel application of probabilistic timed automaton to distinguish musical sequences from non-musical. Our system analyses both audio and video signals to give a classifier that not only extracts musical sequences from movies but identifies their genre. We achieved a recall of 93.24% with precision of 87.34% in song extraction when applied on 10 popular Bollywood movies. An accuracy of 89.5% has been achieved on Bollywood song genre identification.

► Automatic movies song extractor. ► It extracted 69 songs out of 74 from 10 Bollywood movies. ► Video based song genre identification, achieves 89% accuracy.