The unified streaming and non-streaming speech recognition model has achieved
great success due to its comprehensive capabilities. In this paper, we propose
to improve the accuracy of the unified model by bridging the inherent
representation gap between the streaming and non-streaming modes with a
contrastive objective. Specifically, the top-layer hidden representation at the
same frame of the streaming and non-streaming modes are regarded as a positive
pair, encouraging the representation of the streaming mode close to its
non-streaming counterpart. The multiple negative samples are randomly selected
from the rest frames of the same sample under the non-streaming mode.
Experimental results demonstrate that the proposed method achieves consistent
improvements toward the unified model in both streaming and non-streaming
modes. Our method achieves CER of 4.66% in the streaming mode and CER of 4.31%
in the non-streaming mode, which sets a new state-of-the-art on the AISHELL-1
benchmark.

本论文通过对比目标法，缩小流式模式和非流式模式间的固有差距，提高了统一模型的准确性，在 AISHELL-1 基准测试中达到了 CER 值 4.66%（流式模式）和 4.31%（非流式模式），创造了新的最高记录。