Modelling non-stationary noise with spectral factorisation in automatic speech recognition

Speech recognition systems intended for everyday use must be able to cope with a large variety of noise types and levels, including highly non-stationary multi-source mixtures. This study applies spectral factorisation algorithms and long temporal context for separating speech and noise from mixed signals. To adapt the system to varying environments, noise models are acquired from the context, or learnt from the mixture itself without prior information. We also propose methods for reducing the size of the bases used for speech and noise modelling by 20–40 times for better practical applicability. We evaluate the performance of the methods both as a standalone classifier and as a signal-enhancing front-end for external recognisers. For the CHiME noisy speech corpus containing non-stationary multi-source household noises at signal-to-noise ratios ranging from +9 to −6 dB, we report average keyword recognition rates up to 87.8% using a single-stream sparse classification algorithm.

► Non-negative spectro-temporal factorisation is used for robust speech recognition.
► A long window spanning 200–250 ms provides high robustness.
► Noise is modelled adaptively from the context or from the mixture itself.
► Model complexity is reduced significantly by more compact dictionaries.
► The models are suitable both for signal enhancement and direct classification.