Semi-supervised active learning for support vector machines: A novel approach that exploits structure information in data

In today's information society more and more data emerges, e.g., in social networks, technical applications, or business practice. Companies try to commercialize these data using data mining or machine learning methods. For this purpose, the data are often categorized or classified, but many times at high (monetary or temporal) cost. An effective approach to reduce these cost is to apply any kind of active learning (AL) methods, as AL controls the training process of a classifier by specifically querying individual data points (samples), which are then labeled (e.g., provided with class memberships) by a domain expert. However, an analysis of current AL research shows that AL still has some shortcomings. In particular, structure information given by the spatial pattern of the (un)labeled data in the input space of a classification (e.g., cluster information), is used in an insufficient way. To meet this challenge, this article presents a new approach for AL based on support vector machines (SVM) for classification. Structure information is captured by means of probabilistic models that are iteratively improved at run-time when label information becomes available. The probabilistic models are then considered in a selection strategy based on distance, density, diversity, and distribution information for AL (4DS strategy) and in a particular kernel function for SVM (Responsibility Weighted Mahalanobis kernel). With 20 benchmark data sets and with the MNIST data set it is shown that our new solution yields significantly better results than state-of-the-art methods.