HMM-based unit selection speech synthesis using log likelihood ratios derived from perceptual data

• We present a method to make use of perceptual data during the construction of a unit selection speech synthesis system.
• The perceptual data is collected by judging the naturalness of each synthetic prosodic word manually.
• Log likelihood ratios (LLR) are derived from the perceptual data and act as target cost functions in the HMM-based unit selection speech synthesis.
• Several different ways of utilizing LLRs at synthesis time are proposed and compared in our experiments.

This paper presents a hidden Markov model (HMM) based unit selection speech synthesis method using log likelihood ratios (LLR) derived from perceptual data. The perceptual data is collected by judging the naturalness of each synthetic prosodic word manually. Two acoustic models which represent the natural speech and the unnatural synthetic speech are trained respectively. At synthesis time, the LLRs are derived from the estimated acoustic models and integrated into the unit selection criterion as target cost functions. The experimental results show that our proposed method can synthesize more natural speech than the conventional method using likelihood functions. Due to the inadequacy of the acoustic model estimated for the unnatural synthetic speech, utilizing the LLR-based target cost functions to rescore the pre-selection results or the N-best sequences can achieve better performance than substituting them for the original target cost functions directly.