A Multi-Space Distribution (MSD) and two-stream tone modeling approach to Mandarin speech recognition

Tone plays an important role in recognizing spoken tonal languages like Chinese. However, the discontinuity of F0 between voiced and unvoiced transition has traditionally been a hurdle in creating a succinct statistical tone model for automatic speech recognition and synthesis. Various heuristic approaches have been proposed before to get around the problem but with limited success. The Multi-Space Distribution (MSD) proposed by Tokuda et al. which models the two probability spaces, discrete for unvoiced region and continuous for voiced F0 contour, in a linearly weighted mixture, has been successfully applied to Hidden Markov Model (HMM)-based text-to-speech synthesis. We extend MSD to Chinese Mandarin tone modeling for speech recognition. The tone features and spectral features are further separated into two streams and corresponding stream-dependent models are trained. Finally two separated decision trees are constructed by clustering corresponding stream-dependent HMMs. The MSD and two-stream modeling approach is evaluated on large vocabulary, continuously read and spontaneous speech Mandarin databases and its robustness is further investigated in a noisy, continuous Mandarin digit database with eight types of noises at five different SNRs. Experimental results show that our MSD and two-stream based tone modeling approach can significantly improve the recognition performance over a toneless baseline system. The relative tonal syllable error rate (TSER) reductions are 21.0%, 8.4% and 17.4% for large vocabulary read and spontaneous and noisy digit speech recognition tasks, respectively. Comparing with the conventional system where F0 contours are interpolated in unvoiced segments, our approach improves the recognition performance by 9.8%, 7.4% and 13.3% in relative TSER reductions in the corresponding speech recognition tasks, respectively.