Combining heterogeneous deep neural networks with conditional random fields for Chinese dialogue act recognition

•The challenge of dialogue act recognition is discussed.•Multi-modal deep neural networks are combined with conditional random fields for dialogue act recognition.•Numerical experiments are described and good performance is observed.

Dialogue act (DA) recognition is a fundamental step for computers to understand natural-language dialogues because it can reflect the intention of a speaker. However, it is difficult to adapt traditional machine learning models to the dialogue act recognition task due to the heterogeneous features, statistical dependence between the DA tags, and complex relationship between features and the DA tags. In this paper, we propose a new model which combines heterogeneous deep neural networks with conditional random fields (HDNN-CRF) to solve this problem. The proposed model has two main advantages. First, the heterogeneous deep neural networks (HDNN) model, which is extended from the deep neural networks (DNN), retains the powerful ability of representation learning and adds a new skill of dealing with heterogeneous features effectively. Second, the conditional random fields (CRF) can capture the statistical dependence between the DA tags which carries important information to determine the DA tag of the current utterance. To verify the effectiveness of the proposed model, we conduct several experiments on a Chinese corpus, called CASIA-CASSIL corpus. Ten kinds of features are extracted from the utterances. In the experiment, we give some quantitative analysis of these kinds of features. What׳s more, when comparing classification accuracies of the proposed model and some other models, the proposed model has achieved the best performance.