Nonlinear metric learning for kNN and SVMs through geometric transformations

In recent years, research on extending linear metric learning models to handle nonlinear structures has attracted great interests. In this paper, we propose a novel nonlinear solution through the utilization of deformable geometric models to learn spatially varying metrics, and apply the strategy to boost the performance of both kNN and SVM classifiers. Thin-plate splines (TPS) are chosen as the geometric model with the consideration of their remarkable expressive power to generate high-order yet smooth deformations. Through TPS-regulated space transformations, we are able to pull same-class neighbors closer while keeping different-class samples away from each other to improve kNN classification. For SVMs, the same practice is carried out aiming to make the data samples more linearly separable, in the input space or the kernel induced feature space. Improvements in the performance of kNN and SVM classifications are demonstrated through a number of experiments on synthetic and real-world datasets, with comparisons made with several state-of-the-art metric learning solutions.