Training machines to understand natural language and interact with humans is
one of the major goals of artificial intelligence. Recent years have witnessed
an evolution from matching networks to pre-trained language models (PrLMs). In
contrast to the plain-text modeling as the focus of the PrLMs, dialogue texts
involve multiple speakers and reflect special characteristics such as topic
transitions and structure dependencies between distant utterances. However, the
related PrLM models commonly represent dialogues sequentially by processing the
pairwise dialogue history as a whole. Thus the hierarchical information on
either utterance interrelation or speaker roles coupled in such representations
is not well addressed. In this work, we propose compositional learning for
holistic interaction across the utterances beyond the sequential
contextualization from PrLMs, in order to capture the utterance-aware and
speaker-aware representations entailed in a dialogue history. We decouple the
contextualized word representations by masking mechanisms in Transformer-based
PrLM, making each word only focus on the words in current utterance, other
utterances, and two speaker roles (i.e., utterances of sender and utterances of
the receiver), respectively. In addition, we employ domain-adaptive training
strategies to help the model adapt to the dialogue domains. Experimental
results show that our method substantially boosts the strong PrLM baselines in
four public benchmark datasets, achieving new state-of-the-art performance over
previous methods.

本文提出了一种组合学习方法，进一步超越了预先训练的语言模型，以捕捉涉及对话历史的话语感知和说话人感知等表示。通过使用 Transformer-based PrLM 的 masking 机制，将每个单词分别聚焦在当前话语，其他话语和两种说话人角色（即发送者话语和接收者话语）上，有效区分上下文信息。此外，该方法在对话领域适应性训练策略上借鉴了域自适应训练策略，成功地在四个公共基准数据集上取得了新的最先进性能。

基于通道感知的多轮对话理解解耦网络

Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension

Pre-trained language models (PrLMs) have demonstrated superior performance
due to their strong ability to learn universal language representations from
self-supervised pre-training. However, even with the help of the powerful
PrLMs, it is still challenging to effectively capture task-related knowledge
from dialogue texts which are enriched by correlations among speaker-aware
utterances. In this work, we present SPIDER, Structural Pre-traIned DialoguE
Reader, to capture dialogue exclusive features. To simulate the dialogue-like
features, we propose two training objectives in addition to the original LM
objectives: 1) utterance order restoration, which predicts the order of the
permuted utterances in dialogue context; 2) sentence backbone regularization,
which regularizes the model to improve the factual correctness of summarized
subject-verb-object triplets. Experimental results on widely used dialogue
benchmarks verify the effectiveness of the newly introduced self-supervised
tasks.

本文提出了结构化预训练对话阅读器 SPIDER，通过两个训练目标相结合以捕获对话独有的特征，即复原语音顺序和句子骨干规则化，实现了从对话文本中有效地捕获任务相关知识，并在广泛使用的对话基准测试中取得了显著的实验结果。