Recent advancements in data-driven task-oriented dialogue systems (ToDs)
struggle with incremental learning due to computational constraints and
time-consuming issues. Continual Learning (CL) attempts to solve this by
avoiding intensive pre-training, but it faces the problem of catastrophic
forgetting (CF). While generative-based rehearsal CL methods have made
significant strides, generating pseudo samples that accurately reflect the
underlying task-specific distribution is still a challenge. In this paper, we
present Dirichlet Continual Learning (DCL), a novel generative-based rehearsal
strategy for CL. Unlike the traditionally used Gaussian latent variable in the
Conditional Variational Autoencoder (CVAE), DCL leverages the flexibility and
versatility of the Dirichlet distribution to model the latent prior variable.
This enables it to efficiently capture sentence-level features of previous
tasks and effectively guide the generation of pseudo samples. In addition, we
introduce Jensen-Shannon Knowledge Distillation (JSKD), a robust logit-based
knowledge distillation method that enhances knowledge transfer during pseudo
sample generation. Our experiments confirm the efficacy of our approach in both
intent detection and slot-filling tasks, outperforming state-of-the-art
methods.

近期的数据驱动任务导向对话系统在增量学习方面存在计算约束和耗时的问题，而连续学习试图通过避免密集的预训练来解决此问题，但它面临的问题是灾难性遗忘。本文提出了 Dirichlet 连续学习（DCL），这是一种新颖的基于生成的排练策略，用于连续学习。与传统上在条件变分自编码器（CVAE）中使用的高斯潜变量不同，DCL 利用 Dirichlet 分布的灵活性和多功能性来建模潜在先验变量。这使得它能够有效地捕捉先前任务的句子级特征并有效地指导伪样本的生成。此外，我们介绍了 Jensen-Shannon 知识蒸馏（JSKD），一种强大的基于逻辑的知识蒸馏方法，在伪样本生成期间增强了知识传递。我们的实验证实了我们方法在意图检测和槽填充任务上的有效性，并且胜过了最先进的方法。