Maximum distance minimization for feature weighting

• A simple linear programming based feature weighting algorithm is presented.
• The results are independent of the initial scaling of the input dimensions.
• It yields competitive results on UCI datasets.
• The algorithm automatically performs a dimensionality reduction.

We present a new feature weighting method to improve k-Nearest-Neighbor (k-NN) classification. The proposed method minimizes the largest distance between equally labeled data tuples, while retaining a minimum distance between data tuples of different classes, with the goal to group equally labeled data together. It can be implemented as a simple linear program, and in contrast to other feature weighting methods, it does not depend on the initial scaling of the data dimensions. Two versions, a hard and a soft one, are evaluated on real-world datasets from the UCI repository. In particular the soft version compares very well with competing methods. Furthermore, an evaluation is done on challenging gene expression data sets, where the method shows its ability to automatically reduce the dimensionality of the data.