In our dynamic world where data arrives in a continuous stream, continual learning enables us to incrementally add new tasks/domains without the need to retrain from scratch. A major challenge in continual learning of language model is catastrophic forgetting, the tendency of models to forget knowledge from previously trained tasks/domains when training on new ones. This paper studies dialog generation under the continual learning setting. We propose a novel method that 1) uses \textit{Text-Mixup} as data augmentation to avoid model overfitting on replay memory and 2) leverages Batch-Nuclear Norm Maximization (BNNM) to alleviate the problem of mode collapse. Experiments on a $37$-domain task-oriented dialog dataset and DailyDialog (a $10$-domain chitchat dataset) demonstrate that our proposed approach outperforms the state-of-the-art in continual learning.

该论文研究了在持续学习环境下的对话生成，并提出了一种新的方法，使用Text-Mixup作为数据增强以避免模型在重播记忆上过拟合，同时利用Batch-Nuclear Norm Maximization（BNNM）来减轻模式坍塌问题。实验证明，我们的方法在持续学习中优于现有的方法。

迈向稳健和多样性：使用文本混合和批量核范数最大化的对话生成中的持续学习