Speaker Diarization (SD) systems are typically audio-based and operate
independently of the ASR system in traditional speech transcription pipelines
and can have speaker errors due to SD and/or ASR reconciliation, especially
around speaker turns and regions of speech overlap. To reduce these errors, a
Lexical Speaker Error Correction (LSEC), in which an external language model
provides lexical information to correct the speaker errors, was recently
proposed. Though the approach achieves good Word Diarization error rate (WDER)
improvements, it does not use any additional acoustic information and is prone
to miscorrections. In this paper, we propose to enhance and acoustically ground
the LSEC system with speaker scores directly derived from the existing SD
pipeline. This approach achieves significant relative WDER reductions in the
range of 25-40% over the audio-based SD, ASR system and beats the LSEC system
by 15-25% relative on RT03-CTS, Callhome American English and Fisher datasets.

通过使用现有的 SD 管道中直接从说话者得分中获取的信息，将 LSEC 系统与口音得分加强和基于口音的 LSEC 系统相结合，相对于基于音频的 SD、ASR 系统，在 RT03-CTS、美国英语 Callhome 和 Fisher 数据集上取得了 25-40% 的相对 WDER 降低，相对于 LSEC 系统，取得了 15-25% 的相对改善。

AG-LSEC: 音频语境词汇发音错误修正

AG-LSEC: Audio Grounded Lexical Speaker Error Correction

In this paper, we introduce DiarizationLM, a framework to leverage large
language models (LLM) to post-process the outputs from a speaker diarization
system. Various goals can be achieved with the proposed framework, such as
improving the readability of the diarized transcript, or reducing the word
diarization error rate (WDER). In this framework, the outputs of the automatic
speech recognition (ASR) and speaker diarization systems are represented as a
compact textual format, which is included in the prompt to an optionally
finetuned LLM. The outputs of the LLM can be used as the refined diarization
results with the desired enhancement. As a post-processing step, this framework
can be easily applied to any off-the-shelf ASR and speaker diarization systems
without retraining existing components. Our experiments show that a finetuned
PaLM 2-S model can reduce the WDER by rel. 25.9% on the Fisher telephone
conversation dataset, and rel. 31% on the Callhome English dataset.

介绍了 DiarizationLM 框架，利用大型语言模型对说话人辨别系统的输出进行后处理，可用于提高辨别的可读性或降低词辨别错误率。实验证明，通过使用经过微调的 PaLM 2-S 模型，可以在 Fisher 电话对话数据集上降低 WDER 约 25.9％，在 Callhome English 数据集上降低约 31％。