Improving HMM speech synthesis of interrogative sentences by pitch track transformations

Modeling interrogative sentence prosody is a challenging task due to the significant variation of questions. Prosody is produced by intonation, intensity and duration features. Intonation clearly identifies the type of question in most European languages. If only limited training data is available from certain sentence types, synthetic intonation often lacks accuracy, richness and detail due to averaging, inherent in statistical approaches. In this paper, we discuss two rule-based solutions to improve intonation of interrogative sentences. The first approach utilizes a pitch prediction algorithm where a rule-set forms the pitch pattern. The output pattern is combined with an HMM generated F0 contour and the resulting pattern is used for speech synthesis. Our second solution uses key points to define a scaling function. The position of the key points is described by a rule-set, and the value at intermediate points is calculated in training time in a data-driven way in order to maintain the characteristics of the speakers' voice. The proposed two hybrid rule-based-HMM systems were evaluated by speech experts and by perceptual tests. Our evaluation shows that both approaches could significantly improve the prosodic representation of questions in our framework without deteriorating the perceived naturalness of synthetic speech. The pitch contours generated by the systems were compared and evaluated according to how well they could reproduce the unique characteristics of different Hungarian interrogative sentence types. The analysis shows that both of the methods could successfully model the required patterns. In a separate perception test the interrogative prosody identification rate of the solutions was also measured. The results demonstrate that applying our proposed methods the identification rate of questions could be improved significantly. Although our work focuses on our mother tongue -Hungarian- the methodology can be extended to other languages.