Heterogeneous multi-output classification by structured conditional risk minimization

Multi-Output Classification (MOC) includes many learning paradigms, such as multi-label learning, multi-class classification, multi-dimensional classification, etc. It has shown that the classification performance can be much promoted if output structure involved can be incorporated into learning. Consequently, its challenges are naturally from (1) modeling the structure between different output variables, (2) modeling the structure within individual output variables (e.g. the discrete values of the variable are ordered), and (3) handling the heterogeneity (embodied in different ranges or types) of different output variables. However, existing works only address the first two challenges, while the heterogeneity is a relatively more inherent character of MOC and brings more spiny challenge to output structure learning. In this paper, we try to propose a novel method with two-stage learning to overcome all challenges. Firstly, we form a new homogeneous output space by a tricky binarization process for the original heterogeneous output space, and impose some linear constraints over the new output variables to match the within-structure of the corresponding original output variable (meaning that the newly-formed output space is explicitly structured). Secondly, to address our newly-formed problem, we propose a novel structured prediction method based on minimizing estimated structured conditional risk functions, which can not only keep the predicting efficient but also embed the implicit structure among output variables into model learning to improve classification accuracy. The evaluations of our method on various MOC datasets show that it achieves the best classification accuracy compared to its baselines.