Spoken conversational question answering (SCQA) requires machines to model
complex dialogue flow given the speech utterances and text corpora. Different
from traditional text question answering (QA) tasks, SCQA involves audio signal
processing, passage comprehension, and contextual understanding. However, ASR
systems introduce unexpected noisy signals to the transcriptions, which result
in performance degradation on SCQA. To overcome the problem, we propose CADNet,
a novel contextualized attention-based distillation approach, which applies
both cross-attention and self-attention to obtain ASR-robust contextualized
embedding representations of the passage and dialogue history for performance
improvements. We also introduce the spoken conventional knowledge distillation
framework to distill the ASR-robust knowledge from the estimated probabilities
of the teacher model to the student. We conduct extensive experiments on the
Spoken-CoQA dataset and demonstrate that our approach achieves remarkable
performance in this task.

该研究提出了一种名为 CADNet 的方法，该方法使用上下文化注意力分层蒸馏技术，从而在 SCQA 任务中更好地抵御 ASR 噪声，最终在 Spoken-CoQA 数据集上实现了显著性能提升。