Continual learning has emerged as an increasingly important challenge across
various tasks, including Spoken Language Understanding (SLU). In SLU, its
objective is to effectively handle the emergence of new concepts and evolving
environments. The evaluation of continual learning algorithms typically
involves assessing the model's stability, plasticity, and generalizability as
fundamental aspects of standards. However, existing continual learning metrics
primarily focus on only one or two of the properties. They neglect the overall
performance across all tasks, and do not adequately disentangle the plasticity
versus stability/generalizability trade-offs within the model. In this work, we
propose an evaluation methodology that provides a unified evaluation on
stability, plasticity, and generalizability in continual learning. By employing
the proposed metric, we demonstrate how introducing various knowledge
distillations can improve different aspects of these three properties of the
SLU model. We further show that our proposed metric is more sensitive in
capturing the impact of task ordering in continual learning, making it better
suited for practical use-case scenarios.

我们提出了一种评估方法，能够统一评估在连续学习中的稳定性、可塑性和泛化能力，并展示了引入不同的知识蒸馏方法如何改善语音语言理解模型的这三个性质方面。我们进一步展示了我们提出的指标更敏感地捕捉到连续学习中任务顺序的影响，因此更适合实际应用场景。