Natural language generation (NLG) is an essential component of task-oriented dialog systems. Despite the recent success of neural approaches for NLG, they are typically developed in an offline manner for particular domains. To better fit real-life applications where new data come in a stream, we study NLG in a "continual learning" setting to expand its knowledge to new domains or functionalities incrementally. The major challenge towards this goal is catastrophic forgetting, meaning that a continually trained model tends to forget the knowledge it has learned before. To this end, we propose a method called ARPER (Adaptively Regularized Prioritized Exemplar Replay) by replaying prioritized historical exemplars, together with an adaptive regularization technique based on ElasticWeight Consolidation. Extensive experiments to continually learn new domains and intents are conducted on MultiWoZ-2.0 to benchmark ARPER with a wide range of techniques. Empirical results demonstrate that ARPER significantly outperforms other methods by effectively mitigating the detrimental catastrophic forgetting issue.

本篇论文中，作者针对自然语言生成在任务导向型对话系统中的应用，提出了一种连续学习的方法 ARPER（Adaptively Regularized Prioritized Exemplar Replay），通过回放优先历史样本和基于弹性权重整合的自适应正则化技术，有效地缓解了灾难性遗忘问题。在 MultiWoZ-2.0 数据集上的实验证明，ARPER 明显优于其他方法。

面向任务型对话系统的自然语言生成的持续学习