M4L: Maximum margin Multi-instance Multi-cluster Learning for scene modeling

Automatically learning and grouping key motion patterns in a traffic scene captured by a static camera is a fundamental and challenging task for intelligent video surveillance. To learn motion patterns, trajectory obtained by object tracking is parameterized, and scene image is spatially and evenly divided into multiple regular cell blocks which potentially contain several primary motion patterns. Then, for each block, Gaussian Mixture Model (GMM) is adopted to learn its motion patterns based on the parameters of trajectories. Grouping motion pattern can be done by clustering blocks indirectly, and each cluster of blocks corresponds to a certain motion pattern. For one particular block, each of its motion pattern (Gaussian component) can be viewed as an instance, and all motion patterns (Gaussian components) constitute a bag which can correspond to multiple semantic clusters. Therefore, blocks can be grouped as a Multi-instance Multi-cluster Learning (MIMCL) problem, and a novel Maximum Margin Multi-instance Multi-cluster Learning (M4L) algorithm is proposed. To avoid processing a difficult optimization problem, M4L is further relaxed and solved by making use of a combination of the Cutting Plane method and Constrained Concave–Convex Procedure (CCCP). Extensive experiments are conducted on multiple real world video sequences containing various patterns and the results validate the effectiveness of our proposed approach.

► Formulate motion pattern grouping as Multi-instance Multi-cluster Learning problem.
► Propose Maximum Margin Multi-instance Multi-cluster Learning method for clustering.
► Solve the problem by Cutting Plane method and Constrained Concave–Convex Procedure.