Many pretrained multilingual models exhibit cross-lingual transfer ability,
which is often attributed to a learned language-neutral representation during
pretraining. However, it remains unclear what factors contribute to the
learning of a language-neutral representation, and whether the learned
language-neutral representation suffices to facilitate cross-lingual transfer.
We propose a synthetic task, Multilingual Othello (mOthello), as a testbed to
delve into these two questions. We find that: (1) models trained with naive
multilingual pretraining fail to learn a language-neutral representation across
all input languages; (2) the introduction of "anchor tokens" (i.e., lexical
items that are identical across languages) helps cross-lingual representation
alignment; and (3) the learning of a language-neutral representation alone is
not sufficient to facilitate cross-lingual transfer. Based on our findings, we
propose a novel approach - multilingual pretraining with unified output space -
that both induces the learning of language-neutral representation and
facilitates cross-lingual transfer.

预训练的多语言模型在交叉语言迁移方面展现出了能力，本研究旨在探讨学习语言中性表示的因素以及该表示是否足以促进交叉语言迁移。通过引入 “锚定标记”，可以帮助跨语言表示对齐，而仅仅学习语言中性表示是不足以促进交叉语言迁移的。基于研究结果，提出一种新的方法 - 统一输出空间的多语言预训练 - 既能够诱导学习语言中性表示，又能促进交叉语言迁移。

mOthello: 跨语言表示对齐和跨语言迁移在多语言模型中的出现时机

mOthello: When Do Cross-Lingual Representation Alignment and  Cross-Lingual Transfer Emerge in Multilingual Models?

Video-grounded dialogue generation (VDG) requires the system to generate a
fluent and accurate answer based on multimodal knowledge. However, the
difficulty in multimodal knowledge utilization brings serious hallucinations to
VDG models in practice. Although previous works mitigate the hallucination in a
variety of ways, they hardly take notice of the importance of the multimodal
knowledge anchor answer tokens. In this paper, we reveal via perplexity that
different VDG models experience varying hallucinations and exhibit diverse
anchor tokens. Based on this observation, we propose M2K-VDG, a model-adaptive
multimodal knowledge anchor enhancement framework for hallucination reduction.
Furthermore, we introduce the counterfactual effect for more accurate anchor
token detection. The experimental results on three popular benchmarks exhibit
the superiority of our approach over state-of-the-art methods, demonstrating
its effectiveness in reducing hallucinations.

通过计算困惑度，我们揭示了不同的视频对话生成（VDG）模型经历了不同的幻觉，并展示了多样的锚点标记。基于这一观察，我们提出了一种模型适应性的多模态知识锚点增强框架 M2K-VDG，用于减少幻觉产生。进一步，我们引入了反事实效应以更准确地检测锚点标记。在三个流行的基准测试上的实验结果显示，我们的方法优于现有方法，证明了它在减少幻觉方面的有效性。