Multivariate alternating decision trees

Decision trees are comprehensible, but at the cost of a relatively lower prediction accuracy compared to other powerful black-box classifiers such as SVMs. Boosting has been a popular strategy to create an ensemble of decision trees to improve their classification performance, but at the expense of comprehensibility advantage. To this end, alternating decision tree (ADTree) has been proposed to allow boosting within a single decision tree to retain comprehension. However, existing ADTrees are univariate, which limits their applicability. This research proposes a novel algorithm - multivariate ADTree. It presents and discusses its different variations (Fisher׳s ADTree, Sparse ADTree, and Regularized Logistic ADTree) along with their empirical validation on a set of publicly available datasets. It is shown that multivariate ADTree has high prediction accuracy comparable to that of decision tree ensembles, while retaining good comprehension which is close to comprehension of individual univariate decision trees.