This paper presents a streaming speaker-attributed automatic speech
recognition (SA-ASR) model that can recognize ``who spoke what'' with low
latency even when multiple people are speaking simultaneously. Our model is
based on token-level serialized output training (t-SOT) which was recently
proposed to transcribe multi-talker speech in a streaming fashion. To further
recognize speaker identities, we propose an encoder-decoder based speaker
embedding extractor that can estimate a speaker representation for each
recognized token not only from non-overlapping speech but also from overlapping
speech. The proposed speaker embedding, named t-vector, is extracted
synchronously with the t-SOT ASR model, enabling joint execution of speaker
identification (SID) or speaker diarization (SD) with the multi-talker
transcription with low latency. We evaluate the proposed model for a joint task
of ASR and SID/SD by using LibriSpeechMix and LibriCSS corpora. The proposed
model achieves substantially better accuracy than a prior streaming model and
shows comparable or sometimes even superior results to the state-of-the-art
offline SA-ASR model.

提出了一种基于 token 级别序列化输出训练 (t-SOT) 的流式说话者归属性自动语音识别 (SA-ASR) 模型，该模型可以在多人同时说话时实现低延迟的 “谁说了什么” 的识别，并提出了一种基于编码 - 解码的说话者嵌入提取器，可以从非重叠语音和重叠语音中为每个识别的令牌估计说话者表示。