Impacts of machine translation and speech synthesis on speech-to-speech translation

This paper analyzes the impacts of machine translation and speech synthesis on speech-to-speech translation systems. A typical speech-to-speech translation system consists of three components: speech recognition, machine translation and speech synthesis. Many techniques have been proposed for integration of speech recognition and machine translation. However, corresponding techniques have not yet been considered for speech synthesis. The focus of the current work is machine translation and speech synthesis, and we present a subjective evaluation designed to analyze their impact on speech-to-speech translation. The results of these analyses show that the naturalness and intelligibility of the synthesized speech are strongly affected by the fluency of the translated sentences. In addition, several features were found to correlate well with the average fluency of the translated sentences and the average naturalness of the synthesized speech.

► We conducted the subjective evaluation of speech-to-speech translation. ► We analyzed the impacts of machine translation and speech synthesis. ► The fluency is one of the most important factors for speech synthesis systems. ► Some objective measures can be used for predicting the fluency of sentences. ► Some objective measures can be used for predicting the naturalness of speech.