Neural Machine Translation (NMT) models have become successful, but their
performance remains poor when translating on new domains with a limited number
of data. In this paper, we present a novel approach Epi-Curriculum to address
low-resource domain adaptation (DA), which contains a new episodic training
framework along with denoised curriculum learning. Our episodic training
framework enhances the model's robustness to domain shift by episodically
exposing the encoder/decoder to an inexperienced decoder/encoder. The denoised
curriculum learning filters the noised data and further improves the model's
adaptability by gradually guiding the learning process from easy to more
difficult tasks. Experiments on English-German and English-Romanian translation
show that: (i) Epi-Curriculum improves both model's robustness and adaptability
in seen and unseen domains; (ii) Our episodic training framework enhances the
encoder and decoder's robustness to domain shift.

神经机器翻译模型在新领域上的性能仍然较差，本研究提出了一种名为 Epi-Curriculum 的新方法来解决低资源域适应问题，该方法结合了新的经验教学框架和去噪课程学习，通过使编码器 / 解码器与经验不足的解码器 / 编码器进行经验性训练，增强了模型对领域转移的鲁棒性，去噪课程学习则通过逐渐引导学习过程从简单到更困难的任务，进一步提高了模型的适应性。实验结果表明 Epi-Curriculum 提高了模型在可见和不可见领域的鲁棒性和适应性，我们的经验性训练框架增强了编码器和解码器对领域转移的鲁棒性。

Epi-Curriculum: 短期课程学习低资源领域调适在神经网络机器翻译中

Epi-Curriculum: Episodic Curriculum Learning for Low-Resource Domain  Adaptation in Neural Machine Translation

This paper describes our system for the low-resource domain adaptation track
(Track 3) in Spoken Language Understanding Grand Challenge, which is a part of
ICASSP Signal Processing Grand Challenge 2023. In the track, we adopt a
pipeline approach of ASR and NLU. For ASR, we fine-tune Whisper for each domain
with upsampling. For NLU, we fine-tune BART on all the Track3 data and then on
low-resource domain data. We apply masked LM (MLM) -based data augmentation,
where some of input tokens and corresponding target labels are replaced using
MLM. We also apply a retrieval-based approach, where model input is augmented
with similar training samples. As a result, we achieved exact match (EM)
accuracy 63.3/75.0 (average: 69.15) for reminder/weather domain, and won the
1st place at the challenge.

本文介绍了我们在 ICASSP Signal Processing Grand Challenge 2023 的 Spoken Language Understanding Grand Challenge 中低资源领域适应赛道（Track 3）中的系统，其中我们采用 ASR 和 NLU 的流水线方法。我们在每个领域上利用 upampling 对 Whisper 进行微调来进行 ASR。对于 NLU，我们在所有 Track3 数据上进行了 BART 的微调，然后在低资源领域的数据上进行了微调。我们应用了基于 MLM 的遮蔽 LM（MLM）-based 数据增强，并采用了基于检索的方法。通过以上方法，我们在 reminder/weather 领域实现了精确匹配（EM）准确率 63.3/75.0（平均：69.15），并在挑战中获得了第一名。

基于 MLM 数据增强的 ASR 和 NLU 流水线系统：应对低资源挑战

The Pipeline System of ASR and NLU with MLM-based Data Augmentation  toward STOP Low-resource Challenge

Task-oriented semantic parsing is a critical component of virtual assistants,
which is responsible for understanding the user's intents (set reminder, play
music, etc.). Recent advances in deep learning have enabled several approaches
to successfully parse more complex queries (Gupta et al., 2018; Rongali et
al.,2020), but these models require a large amount of annotated training data
to parse queries on new domains (e.g. reminder, music).
In this paper, we focus on adapting task-oriented semantic parsers to
low-resource domains, and propose a novel method that outperforms a supervised
neural model at a 10-fold data reduction. In particular, we identify two
fundamental factors for low-resource domain adaptation: better representation
learning and better training techniques. Our representation learning uses BART
(Lewis et al., 2019) to initialize our model which outperforms encoder-only
pre-trained representations used in previous work. Furthermore, we train with
optimization-based meta-learning (Finn et al., 2017) to improve generalization
to low-resource domains. This approach significantly outperforms all baseline
methods in the experiments on a newly collected multi-domain task-oriented
semantic parsing dataset (TOPv2), which we release to the public.

本文介绍了一种新方法，使用 BART 进行表征学习并使用基于优化的元学习方法来提高到低资源域的泛化性能，以使任务导向的语义解析器适应低资源域，这种方法在新收集的多领域任务导向语义解析数据集（TOPv2）方面明显优于所有基线方法。