A novel temporal fine structure-based speech synthesis model for cochlear implant

It has widely been recognized that delivering the temporal fine structure information to the cochlear implant (CI) users might significantly improve their speech perception performance and lead a breakthrough for the CI design. This paper introduces a new speech synthesis model incorporating the temporal fine structure cue for CI. After bandpass filtering the speech signal, band-specific carriers are constructed by placing the high-rate sinusoidal pulses at the peak positions of the fine structures. The carriers are then amplitude-modulated by their envelopes, and summed to generate the synthesized speech. Mandarin-speaking subjects participated in the acoustic simulation experiment by listening to the voices synthesized by the continuous-interleaved-sampling (CIS) processor and the model-based algorithm in their six-band versions. The experimental results indicated that the model-based algorithm produced significant improvements in Mandarin tone identification, subjective assessment of speech intelligibility and speaker recognition. The proposed model should be helpful for the development of novel CI speech processing strategies to improve the speech perception of cochlear implantees, particularly those speaking tonal languages.