Pushing the Limits of Non-Autoregressive Speech Recognition

04/07/2021

∙

by   Edwin G. Ng, et al.

∙

0

∙

share

We combine recent advancements in end-to-end speech recognition to non-autoregressive automatic speech recognition. We push the limits of non-autoregressive state-of-the-art results for multiple datasets: LibriSpeech, Fisher+Switchboard and Wall Street Journal. Key to our recipe, we leverage CTC on giant Conformer neural network architectures with SpecAugment and wav2vec2 pre-training. We achieve 1.8 5.1 a language model.