Given multiple source word embeddings learnt using diverse algorithms and
lexical resources, meta word embedding learning methods attempt to learn more
accurate and wide-coverage word embeddings.
Prior work on meta-embedding has repeatedly discovered that simple vector
concatenation of the source embeddings to be a competitive baseline.
However, it remains unclear as to why and when simple vector concatenation
can produce accurate meta-embeddings.
We show that weighted concatenation can be seen as a spectrum matching
operation between each source embedding and the meta-embedding, minimising the
pairwise inner-product loss.
Following this theoretical analysis, we propose two \emph{unsupervised}
methods to learn the optimal concatenation weights for creating meta-embeddings
from a given set of source embeddings.
Experimental results on multiple benchmark datasets show that the proposed
weighted concatenated meta-embedding methods outperform previously proposed
meta-embedding learning methods.

本文研究了元词向量嵌入（meta-embedding）的学习方法，提出了利用加权拼接来学习更准确和广泛覆盖面的词向量的两种无监督方法，并在多个基准数据集上进行试验，结果表明这些加权拼接的 meta-embedding 方法优于以前的 meta-embedding 学习方法。

通过非监督加权合并源嵌入学习元词嵌入

Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddings

Word embeddings are trained to predict word cooccurrence statistics, which
leads them to possess different lexical properties (syntactic, semantic, etc.)
depending on the notion of context defined at training time. These properties
manifest when querying the embedding space for the most similar vectors, and
when used at the input layer of deep neural networks trained to solve
downstream NLP problems. Meta-embeddings combine multiple sets of differently
trained word embeddings, and have been shown to successfully improve intrinsic
and extrinsic performance over equivalent models which use just one set of
source embeddings. We introduce word prisms: a simple and efficient
meta-embedding method that learns to combine source embeddings according to the
task at hand. Word prisms learn orthogonal transformations to linearly combine
the input source embeddings, which allows them to be very efficient at
inference time. We evaluate word prisms in comparison to other meta-embedding
methods on six extrinsic evaluations and observe that word prisms offer
improvements in performance on all tasks.

本研究提出了一种简单且高效的元嵌入方法 ——Word Prisms，它可以学习如何根据当前处理文本的任务动态地组合多个不同的词向量训练模型，该方法在 NLP 任务上表现出了很好的性能。

使用词棱镜学习高效的任务特定元嵌入

Learning Efficient Task-Specific Meta-Embeddings with Word Prisms

Click-through rate (CTR) prediction has been one of the most central problems
in computational advertising. Lately, embedding techniques that produce
low-dimensional representations of ad IDs drastically improve CTR prediction
accuracies. However, such learning techniques are data demanding and work
poorly on new ads with little logging data, which is known as the cold-start
problem.
In this paper, we aim to improve CTR predictions during both the cold-start
phase and the warm-up phase when a new ad is added to the candidate pool. We
propose Meta-Embedding, a meta-learning-based approach that learns to generate
desirable initial embeddings for new ad IDs. The proposed method trains an
embedding generator for new ad IDs by making use of previously learned ads
through gradient-based meta-learning. In other words, our method learns how to
learn better embeddings. When a new ad comes, the trained generator initializes
the embedding of its ID by feeding its contents and attributes. Next, the
generated embedding can speed up the model fitting during the warm-up phase
when a few labeled examples are available, compared to the existing
initialization methods.
Experimental results on three real-world datasets showed that Meta-Embedding
can significantly improve both the cold-start and warm-up performances for six
existing CTR prediction models, ranging from lightweight models such as
Factorization Machines to complicated deep models such as PNN and DeepFM. All
of the above apply to conversion rate (CVR) predictions as well.

本文提出了一种名为 Meta-Embedding 的元学习方法，通过先前学习的广告嵌入来生成新的广告 ID 的初始化嵌入，进而提高了 Click-through rate (CTR) 预测的性能，并通过实验结果证明了 Meta-Embedding 在冷启动和预热阶段都能显著提高 CTR 预测模型的精度。