Dynamic texture representation using a deep multi-scale convolutional network

This work addresses dynamic texture representation and recognition via a convolutional multilayer architecture. The proposed method considers an image sequence as a concatenation of spatial images along the time axis as well as spatio-temporal images along both horizontal and vertical axes of an image sequence and uses multilayer convolutional operations to describe each plane. The filters used are learned via principal component analysis (PCA) on each of the three orthogonal planes of an image sequence. A particularly advantageous attribute of the technique is the unsupervised training procedure of the proposed network. An inter-database evaluation has been performed to investigate the generalisation capability of the proposed approach. Moreover, a multi-scale extension of the proposed architecture is presented to capture texture details at multiple resolutions. Through extensive evaluations on different databases, it is shown that the proposed PCA-based network on three orthogonal planes (PCANet-TOP) yields very discriminative features for dynamic texture classification.