This paper describes our system for the SemEval2022 task of matching
dictionary glosses to word embeddings. We focus on the Reverse Dictionary Track
of the competition, which maps multilingual glosses to reconstructed vector
representations. More specifically, models convert the input of sentences to
three types of embeddings: SGNS, Char, and Electra. We propose several
experiments for applying neural network cells, general multilingual and
multitask structures, and language-agnostic tricks to the task. We also provide
comparisons over different types of word embeddings and ablation studies to
suggest helpful strategies. Our initial transformer-based model achieves
relatively low performance. However, trials on different retokenization
methodologies indicate improved performance. Our proposed Elmobased monolingual
model achieves the highest outcome, and its multitask, and multilingual
varieties show competitive results as well.

本文介绍了我们针对 SemEval2022 词典义项匹配任务所设计的系统，主要集中在比赛的 Reverse Dictionary Track 上，涉及到多语言义项到重构向量的映射，主要是将输入句子转换为三类不同类型的嵌入。我们提出了几个实验，包括神经网络单元、通用多语言和多任务结构，以及语言无关技巧。最终我们的 Elmobased 单语言模型表现最佳，并且多任务、多语言变体也表现出竞争力。

在 Semeval-2022 任务 1 中的 1Cademy：通过多语言、多任务和语言无关技巧，研究反向词典任务的有效性

1Cademy at Semeval-2022 Task 1: Investigating the Effectiveness of Multilingual, Multitask, and Language-Agnostic Tricks for the Reverse Dictionary Task

Vision-and-language tasks are gaining popularity in the research community,
but the focus is still mainly on English. We propose a pipeline that utilizes
English-only vision-language models to train a monolingual model for a target
language. We propose to extend OSCAR+, a model which leverages object tags as
anchor points for learning image-text alignments, to train on visual question
answering datasets in different languages. We propose a novel approach to
knowledge distillation to train the model in other languages using parallel
sentences. Compared to other models that use the target language in the
pretraining corpora, we can leverage an existing English model to transfer the
knowledge to the target language using significantly lesser resources. We also
release a large-scale visual question answering dataset in Japanese and Hindi
language. Though we restrict our work to visual question answering, our model
can be extended to any sequence-level classification task, and it can be
extended to other languages as well. This paper focuses on two languages for
the visual question answering task - Japanese and Hindi. Our pipeline
outperforms the current state-of-the-art models by a relative increase of 4.4%
and 13.4% respectively in accuracy.

本文提出了一种使用英文模型作为基础，在日语和印地语等其他语言上进行视觉问题回答的模型训练方法，并采用并行句子的知识蒸馏方法，相较于使用目标语言进行预训练的模型，大大减少了资源开销。通过构建大规模视觉问题回答数据集，实现了比现有技术提高了 4.4％和 13.4％的准确性。

使用知识蒸馏的跨语言视觉语言模型训练 (cViL)

cViL: Cross-Lingual Training of Vision-Language Models using Knowledge Distillation

State-of-the-art unsupervised multilingual models (e.g., multilingual BERT)
have been shown to generalize in a zero-shot cross-lingual setting. This
generalization ability has been attributed to the use of a shared subword
vocabulary and joint training across multiple languages giving rise to deep
multilingual abstractions. We evaluate this hypothesis by designing an
alternative approach that transfers a monolingual model to new languages at the
lexical level. More concretely, we first train a transformer-based masked
language model on one language, and transfer it to a new language by learning a
new embedding matrix with the same masked language modeling objective, freezing
parameters of all other layers. This approach does not rely on a shared
vocabulary or joint training. However, we show that it is competitive with
multilingual BERT on standard cross-lingual classification benchmarks and on a
new Cross-lingual Question Answering Dataset (XQuAD). Our results contradict
common beliefs of the basis of the generalization ability of multilingual
models and suggest that deep monolingual models learn some abstractions that
generalize across languages. We also release XQuAD as a more comprehensive
cross-lingual benchmark, which comprises 240 paragraphs and 1190
question-answer pairs from SQuAD v1.1 translated into ten languages by
professional translators.

新颖的跨语言转移学习方法 - 从单语言模型到新语言，通过学习一个新的词嵌入矩阵来实现，该方法与现有不需要共享词汇表或联合训练的最先进的无监督多语言模型的跨语言分类基准测试表现相似。