Dialogue disentanglement aims to detach the chronologically ordered
utterances into several independent sessions. Conversation utterances are
essentially organized and described by the underlying discourse, and thus
dialogue disentanglement requires the full understanding and harnessing of the
intrinsic discourse attribute. In this paper, we propose enhancing dialogue
disentanglement by taking full advantage of the dialogue discourse
characteristics. First of all, \textbf{in feature encoding stage}, we construct
the heterogeneous graph representations to model the various dialogue-specific
discourse structural features, including the static speaker-role structures
(i.e., speaker-utterance and speaker-mentioning structure) and the dynamic
contextual structures (i.e., the utterance-distance and partial-replying
structure). We then develop a structure-aware framework to integrate the rich
structural features for better modeling the conversational semantic context.
Second, \textbf{in model learning stage}, we perform optimization with a
hierarchical ranking loss mechanism, which groups dialogue utterances into
different discourse levels and carries training covering pair-wise and
session-wise levels hierarchically. Third, \textbf{in inference stage}, we
devise an easy-first decoding algorithm, which performs utterance pairing under
the easy-to-hard manner with a global context, breaking the constraint of
traditional sequential decoding order. On two benchmark datasets, our overall
system achieves new state-of-the-art performances on all evaluations. In-depth
analyses further demonstrate the efficacy of each proposed idea and also reveal
how our methods help advance the task. Our work has great potential to
facilitate broader multi-party multi-thread dialogue applications.

本文提出了一种基于异构图表示的对话解缠方法，构建了不同的讨论结构特征模型，包括静态发言角色结构和动态背景结构，并采用层级排名损失和易于首先解码算法来实现话语理解和分离。

重新审视话语论以进行对话分离

Revisiting Conversation Discourse for Dialogue Disentanglement

Knowledge distillation (KD) is a promising technique for model compression in
neural machine translation. However, where the knowledge hides in KD is still
not clear, which may hinder the development of KD. In this work, we first
unravel this mystery from an empirical perspective and show that the knowledge
comes from the top-1 predictions of teachers, which also helps us build a
potential connection between word- and sequence-level KD. Further, we point out
two inherent issues in vanilla word-level KD based on this finding. Firstly,
the current objective of KD spreads its focus to whole distributions to learn
the knowledge, yet lacks special treatment on the most crucial top-1
information. Secondly, the knowledge is largely covered by the golden
information due to the fact that most top-1 predictions of teachers overlap
with ground-truth tokens, which further restricts the potential of KD. To
address these issues, we propose a novel method named \textbf{T}op-1
\textbf{I}nformation \textbf{E}nhanced \textbf{K}nowledge \textbf{D}istillation
(TIE-KD). Specifically, we design a hierarchical ranking loss to enforce the
learning of the top-1 information from the teacher. Additionally, we develop an
iterative KD procedure to infuse more additional knowledge by distilling on the
data without ground-truth targets. Experiments on WMT'14 English-German, WMT'14
English-French and WMT'16 English-Romanian demonstrate that our method can
respectively boost Transformer$_{base}$ students by +1.04, +0.60 and +1.11 BLEU
scores and significantly outperform the vanilla word-level KD baseline.
Besides, our method shows higher generalizability on different teacher-student
capacity gaps than existing KD techniques.

本文研究神经机器翻译中知识蒸馏的技术，发现知识来源于教师的 top-1 预测，进一步提出一种名为 TIE-KD 的方法用于增强知识蒸馏，包含了层次排序损失和迭代蒸馏等措施，实验证明 TIE-KD 优于基准模型，具有更高的潜力和泛化性能。