Evaluation of synthetic and natural Mandarin visual speech: Initial consonants, single vowels, and syllables

Although the auditory aspects of Mandarin speech are relatively more heavily-researched and well-known in the field, this study addresses its visual aspects by examining the perception of both Mandarin natural and synthetic visual speech. In perceptual experiments, the synthetic visual speech of a computer-animated Mandarin talking head was evaluated and subsequently improved. Also, the basic (or “minimum”) units of Mandarin visual speech were determined for initial consonants and final single-vowels. Overall, the current study achieved solid improvements of synthetic visual speech, and this was one step towards a Mandarin synthetic talking head with realistic speech.