A probabilistic approach for semi-supervised nearest neighbor classification

In supervised classification, we learn from a training set of labeled observations to form a decision rule for classifying all unlabeled test cases. But if the training sample is small, one may fail to extract sufficient information from that sample to develop a good classifier. Because of the statistical instability of nonparametric methods, this problem becomes more evident in the case of nonparametric classification. In such cases, if one can extract useful information also from unlabeled test cases and use that to modify the classification rule, the performance of the resulting classifier can be improved substantially. In this article, we use a probabilistic framework to develop such methods for nearest neighbor classification. The resulting classifiers, called semi-supervised or transductive classifiers, usually perform better than supervised methods, especially when the training sample is small. Some benchmark data sets are analyzed to show the utility of these proposed methods.

► Statistical methods for semi-supervised nearest neighbor classification are developed under a probabilistic framework.
► Results for various choices of tuning parameters are combined using the Bayesian model averaging technique.
► The proposed methods outperform supervised nearest neighbor classifiers, especially when the training sample is small.