Improving domain action classification in goal-oriented dialogues using a mutual retraining method

• Mutual retraining method for domain action classification.
• Re-training the two tightly associated models.
• Performance improvements; higher accuracy of 0.6% and higher macro F1-measure of 1.2%.

To reduce lengthy and rigid interactions of menu-driven navigation and keyword searches, dialogue systems based on a natural language interface have been developed. Domain action classification is an essential part of a dialogue system because speakers’ intentions are determined through the classification process. Although a domain action consists of a tightly associated speech act and a concept sequence, previous studies have independently dealt with speech acts and concept sequences in order to simplify the models, and this simplification has caused a decrease in performance. A retraining method for improving the domain action classification performance is proposed in order to resolve this problem. The proposed method divides a domain action classification model into a speech act classification model and a concept sequence classification model. The speech act classification model repeatedly uses concept sequence classification model outputs as inputs during training. In the experiments with goal-oriented dialogues, the proposed method exhibited a higher accuracy of 0.6% and higher macro F1-measure of 1.7% compared to the SVM and ME models that dealt with speech acts and concept sequences separately. Based on the experimental results, it was determined that the proposed method can improve the performance of some representative machine learning models for domain action classification.