Using articulatory likelihoods in the recognition of dysarthric speech

Millions of individuals have congenital or acquired neuro-motor conditions that limit control of their muscles, including those that manipulate the vocal tract. These conditions, collectively called dysarthria, result in speech that is very difficult to understand both by human listeners and by traditional automatic speech recognition (ASR), which in some cases can be rendered completely unusable.In this work we first introduce a new method for acoustic-to-articulatory inversion which estimates positions of the vocal tract given acoustics using a nonlinear Hammerstein system. This is accomplished based on the theory of task-dynamics using the TORGO database of dysarthric articulation. Our approach uses adaptive kernel canonical correlation analysis and is found to be significantly more accurate than mixture density networks, at or above the 95% level of confidence for most vocal tract variables.Next, we introduce a new method for ASR in which acoustic-based hypotheses are re-evaluated according to the likelihoods of their articulatory realizations in task-dynamics. This approach incorporates high-level, long-term aspects of speech production and is found to be significantly more accurate than hidden Markov models, dynamic Bayesian networks, and switching Kalman filters.

► The TORGO database of dysarthric articulation measures points in the vocal tract.
► Adaptive kernel canonical correlation for acoustic-articulatory inversion.
► Re-ranking hypotheses according to likelihoods of their articulatory realizations.
► Articulatory dynamic Bayes networks are more accurate than hidden Markov models.
► Experiments applied to dysarthric and non-dysarthric speakers.