Manifold Mapping Machine

Nonlinear classification has been a non-trivial task in machine learning for the past decades. In recent years, kernel machines have successfully generalized the inner-product based linear classifiers to nonlinear ones by transforming data into some high or infinite dimensional feature space. However, due to their implicit space transformation and unobservable latent feature space, it is hard to have an intuitive understanding of their working mechanism. In this paper, we propose a comprehensible framework for nonlinear classifier design, called Manifold Mapping Machine (M3). M3 can generalize any linear classifier to nonlinear by transforming data into some low-dimensional feature space explicitly. To demonstrate the effectiveness of M3 framework, we further present an algorithmic implementation of M3 named Supervised Spectral Space Classifier (S3C). Compared with the kernel classifiers, S3C can achieve similar or even better data separation by mapping data into the low-dimensional spectral space, allowing both of its mapped data and new feature space to be examined directly. Moreover, with the discriminative information integrated into the spectral space transformation, the classification performance of S3C is more robust than that of the kernel classifiers. Experimental results show that S3C is superior to other state-of-the-art nonlinear classifiers on both synthetic and real-world data sets.