In this work, we study whether multilingual language models (MultiLMs) can
transfer logical reasoning abilities to other languages when they are
fine-tuned for reasoning in a different language. We evaluate the cross-lingual
reasoning abilities of MultiLMs in two schemes: (1) where the language of the
context and the question remain the same in the new languages that are tested
(i.e., the reasoning is still monolingual, but the model must transfer the
learned reasoning ability across languages), and (2) where the language of the
context and the question is different (which we term code-switched reasoning).
On two logical reasoning datasets, RuleTaker and LeapOfThought, we demonstrate
that although MultiLMs can transfer reasoning ability across languages in a
monolingual setting, they struggle to transfer reasoning abilities in a
code-switched setting. Following this observation, we propose a novel attention
mechanism that uses a dedicated set of parameters to encourage cross-lingual
attention in code-switched sequences, which improves the reasoning performance
by up to 14% and 4% on the RuleTaker and LeapOfThought datasets, respectively.

本研究探讨了多语言语言模型（MultiLMs）在针对不同语言推理时，是否能够将逻辑推理能力转移到其他语言。通过在两种方案中评估 MultiLMs 的跨语言推理能力，我们发现在单语言设置下，MultiLMs 可以在语言之间传递推理能力，但在混合代码推理的情况下，它们很难传递推理能力。基于此观察，我们提出了一种新颖的注意机制，利用专门的参数集在混合代码序列中鼓励跨语言注意力，从而在 RuleTaker 和 LeapOfThought 数据集上分别将推理性能提高了 14% 和 4%。