We propose a novel approach to optimizing a byte-level representation for
end-to-end automatic speech recognition (ASR). Byte-level representation is
often used by large scale multilingual ASR systems when the character set of
the supported languages is large. The compactness and universality of
byte-level representation allow the ASR models to use smaller output
vocabularies and therefore, provide more flexibility. UTF-8 is a commonly used
byte-level representation for multilingual ASR, but it is not designed to
optimize machine learning tasks directly. By using auto-encoder and vector
quantization, we show that we can optimize a byte-level representation for ASR
and achieve better accuracy. Our proposed framework can incorporate information
from different modalities, and provides an error correction mechanism. In an
English/Mandarin dictation task, we show that a bilingual ASR model built with
this approach can outperform UTF-8 representation by 5% relative in error rate.

我们提出了一种对字节级表示进行优化的新方法，通过使用自编码器和向量量化，在自动语音识别中实现更好的准确性。