Tensor-based locally maximum margin classifier for image and video classification

In the past decades, maximum margin classifiers have been recognized as a powerful tool for pattern classification problem. However, for image and video data, it is still hard to map the visual features to the semantic labels, although many maximum margin techniques have announced impressive classification results in their own datasets. The performance uncertainty of maximum margin classifiers in real world applications is mainly caused by the lack of theoretical ground for kernel projection, which unfortunately, is just the key of the performance improvement. Either the kernel space distorts the semantics of the visual space or the dataset is not linear separable in the kernel space, the classification performance cannot be guaranteed. Therefore, this paper proposes a novel Tensor-based Locally Maximum Margin Classifier (TLMMC), which avoids the kernel projection with unclear physical meaning. TLMMC employs the tensor techniques to represent the image and video data in high-order space directly, so the intrinsic structure of the data point is well preserved. To address the difficulty of seeking a linear discriminant hyperplane in the tensor space, TLMMC utilizes a set of local multilinear hyperplanes to construct a global nonlinear one. To find the optimal solution for each local hyperplane, maximum margin technique is used to minimize the generalization error in the smaller but more relevant subset of the training data. Integrating the attractive characters of tensor representation, local-based method, and maximum margin classifier, TLMMC shows impressive results in both theoretical analysis and empirical validation. Besides an effective image and video classifier, this paper also illustrates a new thought of semantics oriented classifier design, which emphasizes the model with clear physical explanation and solid theoretical support instead of performance improvement on individual learning tasks.

Research highlights
► This paper proposes a novel image and video classification algorithm named Tensor-based Locally Maximum Margin Classifier (TLMMC).
► Integrating the attractive characters of tensor representation, local-based method, and maximum margin classifier, TLMMC shows impressive results in both theoretical analysis and empirical validation.
► More importantly, this paper emphasizes a more understandable model instead of a more accurate classifier for individual learning task.
► With clear physical meaning and solid theoretical support, the performance improvement is reliable and the universality of the algorithm is convictive.