Although pre-trained language models (PLMs) have achieved great success and
become a milestone in NLP, abstractive conversational summarization remains a
challenging but less studied task. The difficulty lies in two aspects. One is
the lack of large-scale conversational summary data. Another is that applying
the existing pre-trained models to this task is tricky because of the
structural dependence within the conversation and its informal expression, etc.
In this work, we first build a large-scale (11M) pretraining dataset called
RCS, based on the multi-person discussions in the Reddit community. We then
present TANet, a thread-aware Transformer-based network. Unlike the existing
pre-trained models that treat a conversation as a sequence of sentences, we
argue that the inherent contextual dependency among the utterances plays an
essential role in understanding the entire conversation and thus propose two
new techniques to incorporate the structural information into our model. The
first is thread-aware attention which is computed by taking into account the
contextual dependency within utterances. Second, we apply thread prediction
loss to predict the relations between utterances. We evaluate our model on four
datasets of real conversations, covering types of meeting transcripts,
customer-service records, and forum threads. Experimental results demonstrate
that TANET achieves a new state-of-the-art in terms of both automatic
evaluation and human judgment.

在 NLP 领域中，尽管预训练语言模型（PLMs）取得了巨大的成功，并成为了一个里程碑，但抽象对话摘要仍然是一个具有挑战性但研究较少的任务。本文中，作者们通过在 Reddit 社区中的多人讨论中构建一个大规模（11M）的预训练数据集 RCS，提出了 TANet，一个基于 Transformer 的网络，借助线程感知的关注机制和线程预测损失函数，有效地解决目前模型的局限性，取得了新的最优效果。