Learning by abstraction: Hierarchical classification model using evidential theoretic approach and Bayesian ensemble model

Abstraction is one of the powerful basic techniques for solving complex problems. In this paper we use abstraction along with hierarchical learning to propose a new classification model which is called “Learning by Abstraction (LA)”. The key idea in LA is to apply both supervised and unsupervised learning algorithms for solving complex classification problems. In addition, the proposed model can be useful in semi-supervised learning problems in which we just know the high level category of some training instances. In the learning mode of the proposed model, we find the nearest classes and merge them into a new abstract class. We call the collection of this new abstract class with other existing classes a new abstract level of learning. Then, a new learner is trained to perform the classification task in this abstract level. In the recall mode, in order to classify a new instance we combine the decision of these classifiers using a new classifier ensemble model based on Dempster–Shafer's theory and Bayesian ensemble model.The simulation study results show that the proposed model has two major advantages. First, it can improve the correct classification rate (CCR) of an ordinary classifier, especially in complex classification tasks with high dimensional feature vector and many target classes. Second, the new model is robust to the noise and the rate of CCR improvement of the proposed model increases as the noise level of data goes up. In addition, the proposed model has been examined on a real data set of protein fold pattern recognition problem in which the correct classification rate of the RBF neural network has been improved by about 10%.