Retrieval of movie scenes by semantic matrix and automatic feature weight update

A new semantic-based video scene retrieval method is proposed in this paper. Twelve low-level features extracted from a video clip are represented in a genetic chromosome and target videos that user has in mind are retrieved by the interactive genetic algorithm through the feedback iteration. In this procedure, high-level semantic relevance between retrieved videos is accumulated with so-called semantic relevance matrix and semantic frequency matrix for each iteration, and they are combined with an automatic feature weight update scheme to retrieve more target videos at the next iteration. Experiments over 300 movie scene clips extracted from latest well-known movies, showed an user satisfaction of 0.71 at the fourth iteration for eight queries such as “gloominess”, “happiness”, “quietness”, “action”, “conversation”, “explosion”, “war”, and “car chase”.