Kernel-based feature extraction under maximum margin criterion

In this paper, we study the problem of feature extraction for pattern classification applications. RELIEF is considered as one of the best-performed algorithms for assessing the quality of features for pattern classification. Its extension, local feature extraction (LFE), was proposed recently and was shown to outperform RELIEF. In this paper, we extend LFE to the nonlinear case, and develop a new algorithm called kernel LFE (KLFE). Compared with other feature extraction algorithms, KLFE enjoys nice properties such as low computational complexity, and high probability of identifying relevant features; this is because KLFE is a nonlinear wrapper feature extraction method and consists of solving a simple convex optimization problem. The experimental results have shown the superiority of KLFE over the existing algorithms.

► We proposed a novel feature extraction algorithm KLFE, a generalization of LFE. ► KLFE has the good properties of solving a convex optimization problem. ► We theoretically proved that LFE and KLFE are both basis rotation invariant. ► We implement KLFE via KPCA or KGP followed by LFE.