Two-stage intonation modeling using feedforward neural networks for syllable based text-to-speech synthesis

This paper proposes a two-stage feedforward neural network (FFNN) based approach for modeling fundamental frequency (F0) values of a sequence of syllables. In this study, (i) linguistic constraints represented by positional, contextual and phonological features, (ii) production constraints represented by articulatory features and (iii) linguistic relevance tilt parameters are proposed for predicting intonation patterns. In the first stage, tilt parameters are predicted using linguistic and production constraints. In the second stage, F0 values of the syllables are predicted using the tilt parameters predicted from the first stage, and basic linguistic and production constraints. The prediction performance of the neural network models is evaluated using objective measures such as average prediction error (μ), standard deviation (σ) and linear correlation coefficient (γX,Y). The prediction accuracy of the proposed two-stage FFNN model is compared with other statistical models such as Classification and Regression Tree (CART) and Linear Regression (LR) models. The prediction accuracy of the intonation models is also analyzed by conducting listening tests to evaluate the quality of synthesized speech obtained after incorporation of intonation models into the baseline system. From the evaluation, it is observed that prediction accuracy is better for two-stage FFNN models, compared to the other models.

► Positional, contextual, phonological and articulatory features extracted from the text are proposed for modeling the intonation patterns.
► Tilt parameters derived from the above features are proposed for improving the accuracy of prediction.
► Single-stage and two-stage neural network models are proposed for capturing the intonation patterns from the proposed features.
► Impact of individual features in predicting the intonation patterns is analyzed.
► Explored the influence of duration and intensity constraints on predicting the intonation patterns.