Hierarchical deep neural network for multivariate regression

This paper presents the novel hierarchical deep neural network (HDNN) for the general multivariate regression problem. The recent insight of deep neural network (DNN) is the deep architecture with large training data can bring the best performance in many research areas. The architecture design of our proposed HDNN focuses on both “depth” and “width” of artificial neural network. Specifically for the multivariate regression, HDNN consists of multiple subnets, which is empirically more powerful than DNN by using a divide and conquer strategy. The effectiveness of HDNN as the regression model is verified on two tasks, namely speech enhancement and Chinese handwriting recognition. For the speech enhancement task, our experiments show that HDNN significantly outperforms DNN in terms of perceptual evaluation of speech quality (PESQ), which is an objective measure highly correlated to subjective testing of listening quality. And for Chinese handwriting recognition task, as a nonlinear feature mapping function, we have a very interesting observation that DNN-based approach can not even bring performance gain while HDNN-based approach yields significant improvements of recognition accuracy.