Expressing universal semantics common to all languages is helpful in
understanding the meanings of complex and culture-specific sentences. The
research theme underlying this scenario focuses on learning universal
representations across languages with the usage of massive parallel corpora.
However, due to the sparsity and scarcity of parallel data, there is still a
big challenge in learning authentic ``universals'' for any two languages. In
this paper, we propose EMMA-X: an EM-like Multilingual pre-training Algorithm,
to learn (X)Cross-lingual universals with the aid of excessive multilingual
non-parallel data. EMMA-X unifies the cross-lingual representation learning
task and an extra semantic relation prediction task within an EM framework.
Both the extra semantic classifier and the cross-lingual sentence encoder
approximate the semantic relation of two sentences, and supervise each other
until convergence. To evaluate EMMA-X, we conduct experiments on XRETE, a newly
introduced benchmark containing 12 widely studied cross-lingual tasks that
fully depend on sentence-level representations. Results reveal that EMMA-X
achieves state-of-the-art performance. Further geometric analysis of the built
representation space with three requirements demonstrates the superiority of
EMMA-X over advanced models.

提出 EMMA-X: 一种基于 EM 模型的多语种预训练算法，通过大规模非平行数据学习跨语言的通用语义，实现语义关系预测和跨语言表示学习的统一，取得了最新技术的表现。

EMMA-X：一种 EM 样本的多语言预训练算法，用于跨语言表示学习

EMMA-X: An EM-like Multilingual Pre-training Algorithm for Cross-lingual  Representation Learning

Semantic relation prediction aims to mine the implicit relationships between
objects in heterogeneous graphs, which consist of different types of objects
and different types of links. In real-world scenarios, new semantic relations
constantly emerge and they typically appear with only a few labeled data. Since
a variety of semantic relations exist in multiple heterogeneous graphs, the
transferable knowledge can be mined from some existing semantic relations to
help predict the new semantic relations with few labeled data. This inspires a
novel problem of few-shot semantic relation prediction across heterogeneous
graphs. However, the existing methods cannot solve this problem because they
not only require a large number of labeled samples as input, but also focus on
a single graph with a fixed heterogeneity. Targeting this novel and challenging
problem, in this paper, we propose a Meta-learning based Graph neural network
for Semantic relation prediction, named MetaGS. Firstly, MetaGS decomposes the
graph structure between objects into multiple normalized subgraphs, then adopts
a two-view graph neural network to capture local heterogeneous information and
global structure information of these subgraphs. Secondly, MetaGS aggregates
the information of these subgraphs with a hyper-prototypical network, which can
learn from existing semantic relations and adapt to new semantic relations.
Thirdly, using the well-initialized two-view graph neural network and
hyper-prototypical network, MetaGS can effectively learn new semantic relations
from different graphs while overcoming the limitation of few labeled data.
Extensive experiments on three real-world datasets have demonstrated the
superior performance of MetaGS over the state-of-the-art methods.

提出了一种基于元学习的图神经网络 MetaGS 用于跨异构图的少样本语义关系预测，将图结构分解成多个标准化的子图，并使用双视角图神经网络和超原型网络来捕获和聚合不同子图的局部异构信息和全局结构信息，通过元学习来学习和预测新的关系类型，并在三个真实数据集上取得了良好的效果。