Continual learning has emerged as an increasingly important challenge across
various tasks, including Spoken Language Understanding (SLU). In SLU, its
objective is to effectively handle the emergence of new concepts and evolving
environments. The evaluation of continual learning algorithms typically
involves assessing the model's stability, plasticity, and generalizability as
fundamental aspects of standards. However, existing continual learning metrics
primarily focus on only one or two of the properties. They neglect the overall
performance across all tasks, and do not adequately disentangle the plasticity
versus stability/generalizability trade-offs within the model. In this work, we
propose an evaluation methodology that provides a unified evaluation on
stability, plasticity, and generalizability in continual learning. By employing
the proposed metric, we demonstrate how introducing various knowledge
distillations can improve different aspects of these three properties of the
SLU model. We further show that our proposed metric is more sensitive in
capturing the impact of task ordering in continual learning, making it better
suited for practical use-case scenarios.

我们提出了一种评估方法，能够统一评估在连续学习中的稳定性、可塑性和泛化能力，并展示了引入不同的知识蒸馏方法如何改善语音语言理解模型的这三个性质方面。我们进一步展示了我们提出的指标更敏感地捕捉到连续学习中任务顺序的影响，因此更适合实际应用场景。

评估和改进口语理解中的持续学习

Evaluating and Improving Continual Learning in Spoken Language  Understanding

Graph contrastive learning (GCL) aims to align the positive features while
differentiating the negative features in the latent space by minimizing a
pair-wise contrastive loss. As the embodiment of an outstanding discriminative
unsupervised graph representation learning approach, GCL achieves impressive
successes in various graph benchmarks. However, such an approach falls short of
recognizing the topology isomorphism of graphs, resulting in that graphs with
relatively homogeneous node features cannot be sufficiently discriminated. By
revisiting classic graph topology recognition works, we disclose that the
corresponding expertise intuitively complements GCL methods. To this end, we
propose a novel hierarchical topology isomorphism expertise embedded graph
contrastive learning, which introduces knowledge distillations to empower GCL
models to learn the hierarchical topology isomorphism expertise, including the
graph-tier and subgraph-tier. On top of this, the proposed method holds the
feature of plug-and-play, and we empirically demonstrate that the proposed
method is universal to multiple state-of-the-art GCL models. The solid
theoretical analyses are further provided to prove that compared with
conventional GCL methods, our method acquires the tighter upper bound of Bayes
classification error. We conduct extensive experiments on real-world benchmarks
to exhibit the performance superiority of our method over candidate GCL
methods, e.g., for the real-world graph representation learning experiments,
the proposed method beats the state-of-the-art method by 0.23\% on unsupervised
representation learning setting, 0.43\% on transfer learning setting. Our code
is available at this https URL

通过引入知识蒸馏，我们提出了一种新颖的层次化拓扑同构专家嵌入图对比学习方法，用于增强 GCL 模型学习层次化拓扑同构专业知识，包括图层和子图层。与传统的 GCL 方法相比，我们的方法在贝叶斯分类错误上获得更紧的上界，并在真实世界基准测试中表现出了超越候选 GCL 方法的性能优势。