Recently, neural networks have shown impressive progress across diverse
fields, with speech processing being no exception. However, recent
breakthroughs in this area require extensive offline training using large
datasets and tremendous computing resources. Unfortunately, these models
struggle to retain their previously acquired knowledge when learning new tasks
continually, and retraining from scratch is almost always impractical. In this
paper, we investigate the problem of learning sequence-to-sequence models for
spoken language understanding in a class-incremental learning (CIL) setting and
we propose COCONUT, a CIL method that relies on the combination of experience
replay and contrastive learning. Through a modified version of the standard
supervised contrastive loss applied only to the rehearsal samples, COCONUT
preserves the learned representations by pulling closer samples from the same
class and pushing away the others. Moreover, we leverage a multimodal
contrastive loss that helps the model learn more discriminative representations
of the new data by aligning audio and text features. We also investigate
different contrastive designs to combine the strengths of the contrastive loss
with teacher-student architectures used for distillation. Experiments on two
established SLU datasets reveal the effectiveness of our proposed approach and
significant improvements over the baselines. We also show that COCONUT can be
combined with methods that operate on the decoder side of the model, resulting
in further metrics improvements.

通过结合经验回放和对比学习的组合方法 COCONUT，我们研究了在渐增式学习（CIL）设置中学习序列到序列模型用于口语理解的问题，并通过对基于重放样本的修改的标准监督对比损失，使 COCONUT 通过将相同类别的样本拉近并推开其他样本来保留学习到的表示。此外，我们还利用一种多模态对比损失，通过对齐音频和文本特征帮助模型学习更具辨别性的表示新数据。我们还研究了不同的对比设计，以将对比损失的优势与用于蒸馏的教师 - 学生架构相结合。在两个已建立的 SLU 数据集上的实验证明了我们提出方法的有效性，并且比基准模型有显著的改进。我们还表明 COCONUT 可以与在模型的解码器端操作的方法结合使用，从而进一步改进指标。