An online model for vowel imitation learning

- A virtual infant (LeVI) learns to imitate Finnish vowels in interaction with humans.
- LeVI associates its babbles to imitative responses online with weakly-supervised learning.
- Initially LeVI does not know the number or characteristics of native vowels.
- Initially LeVI does not know the acoustic consequences of its articulations.
- Learning occurs online despite ambiguity in the responses and inaccuracy in babbling.

When infants learn to pronounce speech sounds of their native language, they face the so-called correspondence problem - how to know which articulatory gestures lead to acoustic sounds that are recognized as native speech sounds by other speakers? Previous research suggests that infants might not learn to imitate their parents via autonomous babbling because direct evaluation of the acoustic similarity between the speech sounds of the two is not possible due to different spectral characteristics of the voices caused by differing vocal tract morphologies. We present a novel robust model of infant vowel imitation learning, following a hypothesis that an infant learns to match their productions to their caregiver's speech sounds when the caregiver imitates the infant's babbles. Adapting a cross-situational associative learning technique, evidently present in infant word learning, our simulated language learner can cope with ambiguity in caregiver's responses to babbling as well as with the imprecision of the articulatory gestures of the infant itself. Our fully online learning model also combines vocal exploration and imitative interaction into a single process. Learning performance is evaluated in experiments using Finnish adults as caregivers for a virtual infant, responding to the infant's babbles with lexical words and, after a learning stage, evaluating the quality of the vowels produced by the learner. After 1000 babble-response pairs, our virtual infant is seen to reach a satisfying vowel imitation accuracy of 70-80%.