In this paper, we investigate how the output representation of an end-to-end
neural network affects multilingual automatic speech recognition (ASR). We
study different representations including character-level, byte-level, byte
pair encoding (BPE), and byte-level byte pair encoding (BBPE) representations,
and analyze their strengths and weaknesses. We focus on developing a single
end-to-end model to support utterance-based bilingual ASR, where speakers do
not alternate between two languages in a single utterance but may change
languages across utterances. We conduct our experiments on English and Mandarin
dictation tasks, and we find that BBPE with penalty schemes can improve
utterance-based bilingual ASR performance by 2% to 5% relative even with
smaller number of outputs and fewer parameters. We conclude with analysis that
indicates directions for further improving multilingual ASR.

本文研究了端到端神经网络的输出表征对多语种自动语音识别的影响，并比较了不同类型的表征。我们专注于开发一个单一的端到端模型来支持基于话语的双语 ASR，并在英语和普通话口述任务上进行实验，发现 BBPE 对话语为基础的双语 ASR 性能可以提高 2％到 5％。

双语端到端自动语音识别：基于字节级子词

Bilingual End-to-End ASR with Byte-Level Subwords

Almost all existing machine translation models are built on top of
character-based vocabularies: characters, subwords or words. Rare characters
from noisy text or character-rich languages such as Japanese and Chinese
however can unnecessarily take up vocabulary slots and limit its compactness.
Representing text at the level of bytes and using the 256 byte set as
vocabulary is a potential solution to this issue. High computational cost has
however prevented it from being widely deployed or used in practice. In this
paper, we investigate byte-level subwords, specifically byte-level BPE (BBPE),
which is compacter than character vocabulary and has no out-of-vocabulary
tokens, but is more efficient than using pure bytes only is. We claim that
contextualizing BBPE embeddings is necessary, which can be implemented by a
convolutional or recurrent layer. Our experiments show that BBPE has comparable
performance to BPE while its size is only 1/8 of that for BPE. In the
multilingual setting, BBPE maximizes vocabulary sharing across many languages
and achieves better translation quality. Moreover, we show that BBPE enables
transferring models between languages with non-overlapping character sets.

本篇论文探讨了一种基于字节级子词（BBPE）的机器翻译模型，该模型比字符词汇表更紧凑且没有超出词汇表的令牌，并显示了与 BPE 相当的性能，BBPE 在跨多种语言的情况下最大化词汇共享并实现更好的翻译质量，同时使非重叠字符集的语言之间的模型传递成为可能。