The recently proposed serialized output training (SOT) simplifies
multi-talker automatic speech recognition (ASR) by generating speaker
transcriptions separated by a special token. However, frequent speaker changes
can make speaker change prediction difficult. To address this, we propose
boundary-aware serialized output training (BA-SOT), which explicitly
incorporates boundary knowledge into the decoder via a speaker change detection
task and boundary constraint loss. We also introduce a two-stage connectionist
temporal classification (CTC) strategy that incorporates token-level SOT CTC to
restore temporal context information. Besides typical character error rate
(CER), we introduce utterance-dependent character error rate (UD-CER) to
further measure the precision of speaker change prediction. Compared to
original SOT, BA-SOT reduces CER/UD-CER by 5.1%/14.0%, and leveraging a
pre-trained ASR model for BA-SOT model initialization further reduces
CER/UD-CER by 8.4%/19.9%.

该论文提出了一种基于边界的序列化输出训练（BA-SOT）模型，通过显式地将边界知识纳入解码器，结合语音识别的多个发言者和发言者变化预测，及通过引入二阶段连接时序分类（CTC）策略的令牌级 SOT CTC 来恢复时间上下文信息，并引入了针对发言者变化预测精度的话语依存的字符错误率（UD-CER）。BA-SOT 模型在 CER / UD-CER 方面均优于原始 SOT 模型，并且可以通过预训练的 ASR 模型进行 BA-SOT 模型初始化。