It is very challenging to curate a dataset for language-specific knowledge
and common sense in order to evaluate natural language understanding
capabilities of language models. Due to the limitation in the availability of
annotators, most current multilingual datasets are created through translation,
which cannot evaluate such language-specific aspects. Therefore, we propose
Multilingual CommonsenseQA (mCSQA) based on the construction process of CSQA
but leveraging language models for a more efficient construction, e.g., by
asking LM to generate questions/answers, refine answers and verify QAs followed
by reduced human efforts for verification. Constructed dataset is a benchmark
for cross-lingual language-transfer capabilities of multilingual LMs, and
experimental results showed high language-transfer capabilities for questions
that LMs could easily solve, but lower transfer capabilities for questions
requiring deep knowledge or commonsense. This highlights the necessity of
language-specific datasets for evaluation and training. Finally, our method
demonstrated that multilingual LMs could create QA including language-specific
knowledge, significantly reducing the dataset creation cost compared to manual
creation. The datasets are available at
this https URL

通过语言模型的构建过程，使用 LM 生成问题 / 答案、改进答案和验证 QA，然后减少人工验证的工作量，我们提出了 Multilingual CommonsenseQA（mCSQA），这是一个用于评估多语言 LM 的跨语言语言传递能力的基准数据集。实验结果表明，多语言 LM 在处理容易解决的问题时具有较高的语言传递能力，但处理需要深入知识或常识的问题时传递能力较低。这凸显了对语言特定数据集进行评估和训练的必要性，最后，我们的方法证明了多语言 LM 能够创建包含语言特定知识的 QA，与手动创建相比，显著降低了数据集创建成本。