A comparative study of classifier combination applied to NLP tasks

The paper is devoted to a comparative study of classifier combination methods, which have been successfully applied to multiple tasks including Natural Language Processing (NLP) tasks. There is variety of classifier combination techniques and the major difficulty is to choose one that is the best fit for a particular task. In our study we explored the performance of a number of combination methods such as voting, Bayesian merging, behavior knowledge space, bagging, stacking, feature sub-spacing and cascading, for the part-of-speech tagging task using nine corpora in five languages. The results show that some methods that, currently, are not very popular could demonstrate much better performance. In addition, we learned how the corpus size and quality influence the combination methods performance. We also provide the results of applying the classifier combination methods to the other NLP tasks, such as name entity recognition and chunking. We believe that our study is the most exhaustive comparison made with combination methods applied to NLP tasks so far.

► Our NLP bibliographical analysis shows low balance in combination methods usage.
► We experiment with eight combination methods, 13 corpora, four classifiers, and four NLP tasks.
► Scenarios with low quality data or base classifiers can obtain major improvements.
► Classification can also be improved combining different versions of only one classifier.
► Stacking can combine integrating heterogeneous information from multiple sources.