State-of-the-art ASR systems have achieved promising results by modeling
local and global interactions separately. While the former can be computed
efficiently, global interactions are usually modeled via attention mechanisms,
which are expensive for long input sequences. Here, we address this by
extending HyperMixer, an efficient alternative to attention exhibiting linear
complexity, to the Conformer architecture for speech recognition, leading to
HyperConformer. In particular, multi-head HyperConformer achieves comparable or
higher recognition performance while being more efficient than Conformer in
terms of inference speed, memory, parameter count, and available training data.
HyperConformer achieves a word error rate of 2.9% on Librispeech test-clean
with less than 8M neural parameters and a peak memory during training of 5.7GB,
hence trainable with accessible hardware. Encoder speed is between 38% on
mid-length speech and 56% on long speech faster than an equivalent Conformer.
(The HyperConformer recipe is publicly available in:
this https URL)

本文介绍了一种名为 HyperConformer 的语音识别结构，它通过引入高效的 HyperMixer 机制，实现了对于长输入序列较为经济的全局交互建模，并在可获得的训练数据限制下达到与或高于传统结构 Conformer 相似的识别表现。