Transformer-based pre-trained language models (PLMs) have achieved remarkable
performance in various natural language processing (NLP) tasks. However,
pre-training such models can take considerable resources that are almost only
available to high-resource languages. On the contrary, static word embeddings
are easier to train in terms of computing resources and the amount of data
required. In this paper, we introduce MoSECroT Model Stitching with Static Word
Embeddings for Crosslingual Zero-shot Transfer), a novel and challenging task
that is especially relevant to low-resource languages for which static word
embeddings are available. To tackle the task, we present the first framework
that leverages relative representations to construct a common space for the
embeddings of a source language PLM and the static word embeddings of a target
language. In this way, we can train the PLM on source-language training data
and perform zero-shot transfer to the target language by simply swapping the
embedding layer. However, through extensive experiments on two classification
datasets, we show that although our proposed framework is competitive with weak
baselines when addressing MoSECroT, it fails to achieve competitive results
compared with some strong baselines. In this paper, we attempt to explain this
negative result and provide several thoughts on possible improvement.

使用静态词向量和相对表示构建共同空间进行跨语种零 - shot 传输，虽然提出的模型在处理 MoSECroT 方面与弱基准模型相竞争，但无法与强基准模型相同的结果竞争。