Transfer learning is an exciting area of Natural Language Processing that has
the potential to both improve model performance and increase data efficiency.
This study explores the effects of varying quantities of target task training
data on sequential transfer learning in the dialog domain. We hypothesize that
a model can utilize the information learned from a source task to better learn
a target task, thereby reducing the number of target task training samples
required. Unintuitively, our data shows that often target task training data
size has minimal effect on how sequential transfer learning performs compared
to the same model without transfer learning. Our results lead us to believe
that this unexpected result could be due to the effects of catastrophic
forgetting, motivating further work into methods that prevent such forgetting.

本研究探讨了在对话领域中，基于顺序迁移学习的多少目标任务训练数据会对模型性能和数据效率产生何种影响。研究结果表明，在许多情况下，与没有迁移学习的相同模型相比，目标任务的训练数据大小对顺序迁移学习的表现几乎没有影响。这一出人意料的结果可能是灾难性遗忘效应的影响，进一步的工作需要研究如何避免这种遗忘。