Solving classification problems by knowledge sets

We propose a novel theoretical model and a method for solving binary classification problems. First, we find knowledge sets in the input space by using estimated density functions. Then, we find the final solution outside knowledge sets. We derived bounds for classification error based on knowledge sets. We estimate knowledge sets with examples and find the solution by using support vector machines (SVM). We performed tests on various real world data sets, and we achieved similar generalization performance compared to SVM with significantly smaller number of support vectors.