An empirical evaluation of the performance of binary classifiers in the prediction of credit ratings changes

In this study, we examine the predictive performance of a wide class of binary classifiers using a large sample of international credit ratings changes from the period 1983-2013. Using a number of financial, market, corporate governance, macro-economic and other indicators as explanatory variables, we compare classifiers ranging from conventional techniques (such as logit/probit and LDA) to fully nonlinear classifiers, including neural networks, support vector machines and more recent statistical learning techniques such as generalised boosting, AdaBoost and random forests. We find that the newer classifiers significantly outperform all other classifiers on both the cross sectional and longitudinal test samples; and prove remarkably robust to different data structures and assumptions. Simple linear classifiers such as logit/probit and LDA are found nonetheless to predict quite accurately on the test samples, in some cases performing comparably well to more flexible model structures. We conclude that simpler classifiers can be viable alternatives to more sophisticated approaches, particularly if interpretability is an important objective of the modelling exercise. We also suggest effective ways to enhance the predictive performance of many of the binary classifiers examined in this study.