Prosodic variation enhancement using unsupervised context labeling for HMM-based expressive speech synthesis

• We propose a technique for adding prosodic variations in HMM-based speech synthesis.
• Phrase-level contexts are defined that compensate the lack of prosodic variations.
• Average prosodic difference between original and synthetic speech is used.
• Expressive speech of sales talk and fairy tale is used for evaluation.
• Users can manually change phrase-level pitch without degrading naturalness.

This paper proposes an unsupervised labeling technique using phrase-level prosodic contexts for HMM-based expressive speech synthesis, which enables users to manually enhance prosodic variations of synthetic speech without degrading the naturalness. In the proposed technique, HMMs are first trained using the conventional labels including only linguistic information, and prosodic features are generated from the HMMs. The average difference of original and generated prosodic features for each accent phrase is then calculated and classified into three classes, e.g., low, neutral, and high in the case of fundamental frequency. The created prosodic context label has a practical meaning such as high/low of relative pitch at the phrase level, and hence it is expected that users can modify the prosodic characteristic of synthetic speech in an intuitive way by manually changing the proposed labels. In the experiments, we evaluate the proposed technique in both ideal and practical conditions using speech of sales talk and fairy tale recorded under a realistic domain. In the evaluation under the practical condition, we evaluate whether the users achieve their intended prosodic modification by changing the proposed context label of a certain accent phrase for a given sentence.