Detecting dialogue breakdown in real time is critical for conversational AI
systems, because it enables taking corrective action to successfully complete a
task. In spoken dialog systems, this breakdown can be caused by a variety of
unexpected situations including high levels of background noise, causing STT
mistranscriptions, or unexpected user flows. In particular, industry settings
like healthcare, require high precision and high flexibility to navigate
differently based on the conversation history and dialogue states. This makes
it both more challenging and more critical to accurately detect dialog
breakdown. To accurately detect breakdown, we found it requires processing
audio inputs along with downstream NLP model inferences on transcribed text in
real time. In this paper, we introduce a Multimodal Contextual Dialogue
Breakdown (MultConDB) model. This model significantly outperforms other known
best models by achieving an F1 of 69.27.

实时检测对话中的中断对于会话型人工智能系统至关重要，因为它能采取纠正措施以成功完成任务。本文介绍了一种多模态上下文对话中断模型（MultConDB），该模型通过在实时转录文本上进行音频输入处理和下游自然语言处理模型推理来精确检测对话中断，并获得了 69.27 的 F1 分数，显著优于其他已知最佳模型。