The ability to handle miscommunication is crucial to robust and faithful
conversational AI. People usually deal with miscommunication immediately as
they detect it, using highly systematic interactional mechanisms called repair.
One important type of repair is Third Position Repair (TPR) whereby a speaker
is initially misunderstood but then corrects the misunderstanding as it becomes
apparent after the addressee's erroneous response. Here, we collect and
publicly release Repair-QA, the first large dataset of TPRs in a conversational
question answering (QA) setting. The data is comprised of the TPR turns,
corresponding dialogue contexts, and candidate repairs of the original turn for
execution of TPRs. We demonstrate the usefulness of the data by training and
evaluating strong baseline models for executing TPRs. For stand-alone TPR
execution, we perform both automatic and human evaluations on a fine-tuned T5
model, as well as OpenAI's GPT-3 LLMs. Additionally, we extrinsically evaluate
the LLMs' TPR processing capabilities in the downstream conversational QA task.
The results indicate poor out-of-the-box performance on TPR's by the GPT-3
models, which then significantly improves when exposed to Repair-QA.

处理不正常交流对于鲁棒和忠实的会话型人工智能至关重要。我们收集和公开发布了 Repair-QA，这是第一个大型的用于会话型问答设置中的第三位置修复的数据集。通过训练和评估强大的基准模型来展示数据的有效性，并且结果表明，GPT-3 模型在修复方面的性能从开箱即用时较差，但在接触到 Repair-QA 后显著改善。