Word embedding learning methods require a large number of occurrences of a
word to accurately learn its embedding. However, out-of-vocabulary (OOV) words
which do not appear in the training corpus emerge frequently in the smaller
downstream data. Recent work formulated OOV embedding learning as a few-shot
regression problem and demonstrated that meta-learning can improve results
obtained. However, the algorithm used, model-agnostic meta-learning (MAML) is
known to be unstable and perform worse when a large number of gradient steps
are used for parameter updates. In this work, we propose the use of Leap, a
meta-learning algorithm which leverages the entire trajectory of the learning
process instead of just the beginning and the end points, and thus ameliorates
these two issues. In our experiments on a benchmark OOV embedding learning
dataset and in an extrinsic evaluation, Leap performs comparably or better than
MAML. We go on to examine which contexts are most beneficial to learn an OOV
embedding from, and propose that the choice of contexts may matter more than
the meta-learning employed.

该论文提出了一种利用 Leap 元学习算法来学习 OOV 嵌入的方法，与 MAML 相比，Leap 可以更好地解决梯度方面的问题，并在评估中取得了与 MAML 相当或更好的性能，作者也探究了哪些上下文对学习 OOV 嵌入更有利，并提出了上下文选择可能比使用的元学习更重要。

基于轨迹的元学习：用于未知词嵌入学习

Trajectory-Based Meta-Learning for Out-Of-Vocabulary Word Embedding  Learning

Existing approaches for learning word embeddings often assume there are
sufficient occurrences for each word in the corpus, such that the
representation of words can be accurately estimated from their contexts.
However, in real-world scenarios, out-of-vocabulary (a.k.a. OOV) words that do
not appear in training corpus emerge frequently. It is challenging to learn
accurate representations of these words with only a few observations. In this
paper, we formulate the learning of OOV embeddings as a few-shot regression
problem, and address it by training a representation function to predict the
oracle embedding vector (defined as embedding trained with abundant
observations) based on limited observations. Specifically, we propose a novel
hierarchical attention-based architecture to serve as the neural regression
function, with which the context information of a word is encoded and
aggregated from K observations. Furthermore, our approach can leverage
Model-Agnostic Meta-Learning (MAML) for adapting the learned model to the new
corpus fast and robustly. Experiments show that the proposed approach
significantly outperforms existing methods in constructing accurate embeddings
for OOV words, and improves downstream tasks where these embeddings are
utilized.

本文提出了一种基于少样本回归的新型分层注意力神经网络结构来解决训练集中不存在的词导致词向量精度下降的问题，实验表明该方法在学习 OOV 词嵌入方面具有优越性。