Large-scale open-domain dialogue data crawled from public social media has greatly improved the performance of dialogue models. However, long-turn dialogues are still highly scarce. Specifically, most dialogue sessions in existing corpora have less than three turns. To alleviate this issue, we propose the Retrieve, Reorganize and Rescale framework (Re$^3$Dial), which can automatically construct a billion-scale long-turn dialogue corpus from existing short-turn dialogue data. Re$^3$Dial first trains an Unsupervised Dense Session Retriever (UDSR) to capture semantic and discourse relationships within multi-turn dialogues for retrieving relevant and coherent sessions. It then reorganizes the short-turn dialogues into long-turn sessions via recursively retrieving and selecting the consecutive sessions with our proposed diversity sampling strategy. Extensive evaluations on multiple multi-turn dialogue benchmarks demonstrate that Re$^3$Dial consistently and significantly improves the dialogue model's ability to utilize long-term context for modeling multi-turn dialogues across different pre-training settings. Finally, we build a toolkit for efficiently rescaling dialogue corpus with Re$^3$Dial, which enables us to construct a corpus containing 1B Chinese dialogue sessions with 11.3 turns on average (5X longer than the original EVA corpus). We will release our UDSR model, toolkit, and data for public use.

本文提出了使用机器学习技术构建大规模多轮对话数据集及相应框架Re$^3$Dial，可以将已有短轮对话数据转变成长轮对话数据，尤其适用于处理大规模开放域网络聊天数据，实验证明相比现有技术大幅提高了在不同预训练环境下的多轮对话建模能力。

Re$^3$Dial：用于长轮次开放域对话预训练的检索、重组和缩放对话语料库