Two-stage multiple kernel learning for supervised dimensionality reduction

• A novel approach for non-linear supervised dimensionality reduction is proposed.
• The proposed method uses kernel discriminant analysis method with new kernels.
• A multiple kernel learning (MKL) paradigm produces the new kernel function.
• Proper criteria for supervised dimensionality reduction are optimized in the MKL part.
• The proposed method is successfully evaluated using many real-world datasets.

In supervised dimensionality reduction methods for pattern recognition tasks, the information of the class labels is considered through the process of reducing the input dimensionality, to improve the classification accuracy. Using nonlinear mappings for this purpose makes these models more appropriate for nonlinearly distributed data. In this paper, a new nonlinear supervised dimensionality reduction model is introduced. The dimensionality reduction process in this model is performed through a multiple kernel learning paradigm in two stages. In the first stage, three suitable criteria for supervised dimensionality reduction containing fisher, homoscedasticity, and between-class distance criteria are used to find the kernel weights. With these weights, a linear combination of several valid kernels is computed to make a new suitable kernel function. In the second stage, the kernel discriminant analysis method is employed for nonlinear supervised dimensionality reduction using the kernel computed in the first stage. Many experiments on a variety of real-world datasets including handwritten digits images, objects images, and other datasets, show that the proposed approach among a number of well-known related techniques, results in accurate and fast classifications.