User simulation in a stochastic dialog system

We present a new methodology of user simulation applied to the evaluation and refinement of stochastic dialog systems. Common weaknesses of these systems are the scarceness of the training corpus and the cost of an evaluation made by real users. We have considered the user simulation technique as an alternative way of testing and improving our dialog system. We have developed a new dialog manager that plays the role of the user. This user dialog manager incorporates several knowledge sources, combining statistical and heuristic information in order to define its dialog strategy. Once the user simulator is integrated into the dialog system, it is possible to enhance the dialog models by an automatic strategy learning. We have performed an extensive evaluation, achieving a slight but clear improvement of the dialog system.