Word embedding learning methods require a large number of occurrences of a word to accurately learn its embedding. However, out-of-vocabulary (OOV) words which do not appear in the training corpus emerge frequently in the smaller downstream data. Recent work formulated OOV embedding learning as a few-shot regression problem and demonstrated that meta-learning can improve results obtained. However, the algorithm used, model-agnostic meta-learning (MAML) is known to be unstable and perform worse when a large number of gradient steps are used for parameter updates. In this work, we propose the use of Leap, a meta-learning algorithm which leverages the entire trajectory of the learning process instead of just the beginning and the end points, and thus ameliorates these two issues. In our experiments on a benchmark OOV embedding learning dataset and in an extrinsic evaluation, Leap performs comparably or better than MAML. We go on to examine which contexts are most beneficial to learn an OOV embedding from, and propose that the choice of contexts may matter more than the meta-learning employed.

该论文提出了一种利用 Leap 元学习算法来学习 OOV 嵌入的方法，与 MAML 相比，Leap 可以更好地解决梯度方面的问题，并在评估中取得了与 MAML 相当或更好的性能，作者也探究了哪些上下文对学习OOV嵌入更有利，并提出了上下文选择可能比使用的元学习更重要。

基于轨迹的元学习：用于未知词嵌入学习