Training machines to understand natural language and interact with humans is
one of the major goals of artificial intelligence. Recent years have witnessed
an evolution from matching networks to pre-trained language models (PrLMs). In
contrast to the plain-text modeling as the focus of the PrLMs, dialogue texts
involve multiple speakers and reflect special characteristics such as topic
transitions and structure dependencies between distant utterances. However, the
related PrLM models commonly represent dialogues sequentially by processing the
pairwise dialogue history as a whole. Thus the hierarchical information on
either utterance interrelation or speaker roles coupled in such representations
is not well addressed. In this work, we propose compositional learning for
holistic interaction across the utterances beyond the sequential
contextualization from PrLMs, in order to capture the utterance-aware and
speaker-aware representations entailed in a dialogue history. We decouple the
contextualized word representations by masking mechanisms in Transformer-based
PrLM, making each word only focus on the words in current utterance, other
utterances, and two speaker roles (i.e., utterances of sender and utterances of
the receiver), respectively. In addition, we employ domain-adaptive training
strategies to help the model adapt to the dialogue domains. Experimental
results show that our method substantially boosts the strong PrLM baselines in
four public benchmark datasets, achieving new state-of-the-art performance over
previous methods.

本文提出了一种组合学习方法，进一步超越了预先训练的语言模型，以捕捉涉及对话历史的话语感知和说话人感知等表示。通过使用 Transformer-based PrLM 的 masking 机制，将每个单词分别聚焦在当前话语，其他话语和两种说话人角色（即发送者话语和接收者话语）上，有效区分上下文信息。此外，该方法在对话领域适应性训练策略上借鉴了域自适应训练策略，成功地在四个公共基准数据集上取得了新的最先进性能。

基于通道感知的多轮对话理解解耦网络

Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension

A multi-turn dialogue is composed of multiple utterances from two or more
different speaker roles. Thus utterance- and speaker-aware clues are supposed
to be well captured in models. However, in the existing retrieval-based
multi-turn dialogue modeling, the pre-trained language models (PrLMs) as
encoder represent the dialogues coarsely by taking the pairwise dialogue
history and candidate response as a whole, the hierarchical information on
either utterance interrelation or speaker roles coupled in such representations
is not well addressed. In this work, we propose a novel model to fill such a
gap by modeling the effective utterance-aware and speaker-aware representations
entailed in a dialogue history. In detail, we decouple the contextualized word
representations by masking mechanisms in Transformer-based PrLM, making each
word only focus on the words in current utterance, other utterances, two
speaker roles (i.e., utterances of sender and utterances of receiver),
respectively. Experimental results show that our method boosts the strong
ELECTRA baseline substantially in four public benchmark datasets, and achieves
various new state-of-the-art performance over previous methods. A series of
ablation studies are conducted to demonstrate the effectiveness of our method.

本文提出了一种新型方法，旨在通过建模对话历史中蕴含的有效话语意识和发言人意识表示，从而解决现有基于检索的多轮对话建模中针对言语和说话人角色的细粒度表示不足的问题，并在四个公共基准数据集上显著提高了 ELECTRA 基线的性能，达到了多种新的最新性能。