Whisper to normal speech conversion using pitch estimated from spectrum

• A systematic method to connect a whispered vowel to its perceived pitch is presented.
• A regression function to predict the perceived pitch from the spectral information of the whispered speech is obtained.
• A whispered-to-normal speech converter is constructed and the pitch accents perceived in whispered Japanese words were recovered by the conversion.

We can perceive pitch in whispered speech, although fundamental frequency (F0) does not exist physically or phonetically due to the lack of vocal-fold vibration. This study was carried out to determine how people generate such an unvoiced pitch. We conducted experiments in which speakers uttered five whispered Japanese vowels in accordance with the pitch of a guide pure tone. From the results, we derived a multiple regression function to convert the outputs of a mel-scaled filter bank of whispered speech into the perceived pitch value. Next, using this estimated pitch value as F0, we constructed a system for conversion of whispered speech to normal speech. Since the pitch varies with time according to the spectral shape, it was expected that the pitch accent would be kept by this conversion. Indeed, auditory experiments demonstrated that the correctly perceived rate of Japanese word accent was increased from 55.5% to 72.0% compared with that when a constant F0 was used.