Bilinear dynamics for crowd video analysis

In this paper, a novel crowd descriptor, termed as bilinear CD (Curl and Divergence) descriptor, is proposed based on the bilinear interaction of curl and divergence. Specifically, the curl and divergence activation maps are computed from the normalized average flow. A local curl patch and the corresponding divergence patch are cropped respectively from the activation maps. The outer product of the two local patches is defined as the bilinear CD vector. Through sliding a window on the activation maps, we can get hundreds to thousands local bilinear CD vectors. To encode them into a compact representation, fisher vector pooling and PCA algorithms are applied on the local descriptors. Experiments on the CUHK crowd dataset show that the proposed bilinear dynamics can improve the performance of video classification and retrieval by a noticeable margin when compared with the existing crowd features.