Meta-embedding (ME) learning is an emerging approach that attempts to learn
more accurate word embeddings given existing (source) word embeddings as the
sole input.
Due to their ability to incorporate semantics from multiple source embeddings
in a compact manner with superior performance, ME learning has gained
popularity among practitioners in NLP.
To the best of our knowledge, there exist no prior systematic survey on ME
learning and this paper attempts to fill this need.
We classify ME learning methods according to multiple factors such as whether
they (a) operate on static or contextualised embeddings, (b) trained in an
unsupervised manner or (c) fine-tuned for a particular task/domain.
Moreover, we discuss the limitations of existing ME learning methods and
highlight potential future research directions.

本文为填补学界在 Meta-embedding learning 系统调查领域的空白着重于分类 Meta-embedding learning 方法，并在讨论其限制和未来研究方向时，探究了该方法在多源嵌入语义中的优势。

词元嵌入学习调查

A Survey on Word Meta-Embedding Learning

A variety of contextualised language models have been proposed in the NLP
community, which are trained on diverse corpora to produce numerous Neural
Language Models (NLMs). However, different NLMs have reported different levels
of performances in downstream NLP applications when used as text
representations. We propose a sentence-level meta-embedding learning method
that takes independently trained contextualised word embedding models and
learns a sentence embedding that preserves the complementary strengths of the
input source NLMs. Our proposed method is unsupervised and is not tied to a
particular downstream task, which makes the learnt meta-embeddings in principle
applicable to different tasks that require sentence representations.
Specifically, we first project the token-level embeddings obtained by the
individual NLMs and learn attention weights that indicate the contributions of
source embeddings towards their token-level meta-embeddings. Next, we apply
mean and max pooling to produce sentence-level meta-embeddings from token-level
meta-embeddings. Experimental results on semantic textual similarity benchmarks
show that our proposed unsupervised sentence-level meta-embedding method
outperforms previously proposed sentence-level meta-embedding methods as well
as a supervised baseline.

提出了一种句子级元嵌入学习方法，利用不同的上下文化语言模型，通过无监督的学习方式学习句子嵌入，以满足不同需要的下游自然语言处理任务，实验证明提出的方法优于以往的方法和有监督的基线。