Feature selection based on quality of information

Feature selection as one of the key problems of data preprocessing is a hot research topic in pattern recognition, machine learning, and data mining. Evaluating the relevance between features based on information theory is a popular and effective method. However, very little research pays attention to the distinguishing ability of feature, i.e., the degree of a feature distinguishes a given sample with other samples. In this paper, we propose a new feature selection method based on the distinguishing ability of feature. First, we define the concept of maximum-nearest-neighbor, and use this concept to discriminate the nearest neighbors of samples. Then, we present a new measure method for evaluating the quality of feature. Finally, the proposed algorithm is tested on benchmark datasets. Experimental results show that the proposed algorithm can effectively select a discriminative feature subset, and performs as well as or better than other popular feature selection algorithms.