A unified framework for translation and understanding allowing discriminative joint decoding for multilingual speech semantic interpretation

•A comparison between the methods used for speech translation and understanding.•A unified framework for translation and understanding.•A discriminative joint decoding for multilingual speech semantic interpretation.•The proposition is competitive with state-of-the-art techniques.•The framework can be generalized to other components of a dialogue system.

Probabilistic approaches are now widespread in most natural language processing applications and selection of a particular approach usually depends on the task at hand. Targeting speech semantic interpretation in a multilingual context, this paper presents a comparison between the state-of-the-art methods used for machine translation and speech understanding. This comparison justifies our proposition of a unified framework for both tasks based on a discriminative approach. We demonstrate that this framework can be used to perform a joint translation-understanding decoding which allows to combine, in the same process, translation and semantic tagging scores of a sentence. A cascade of finite-state transducers is used to compose the translation and understanding hypothesis graphs (1-bests, word graphs or confusion networks). Not only this proposition is competitive with the state-of-the-art techniques but also its framework is even more attractive as it can be generalized to other components of human–machine vocal interfaces (e.g. speech recognizer) so as to allow a richer transmission of information between them.