So near and yet so far: New insight into properties of some well-known classifier paradigms

This article provides some new insight into the properties of four well-established classifier paradigms, namely support vector machines (SVM), classifiers based on mixture density models (CMM), fuzzy classifiers (FCL), and radial basis function neural networks (RBF). It will be shown that these classifiers can be formulated in a way such that they are functionally equivalent or at least highly similar. The interpretation of a specific classifier as being an SVM, CMM, FCL, or RBF then only depends on the objective function and the optimization algorithm used to adjust the parameters. The properties of these four paradigms, however, are very different: a discriminative classifier such as an SVM is expected to have optimal generalization capabilities on new data, a generative classifier such as a CMM also aims at modeling the processes from which the observed data originate, and a comprehensible classifier such as an FCL is intended to be parameterized and understood by human domain experts. We will discuss the advantages and disadvantages of these properties and show how they can be measured numerically in order to compare these classifiers. In such a way, the article aims at supporting a practitioner in assessing the properties of classifier paradigms and in selecting or combining certain paradigms for a given application problem.