Generalization and robustness to input variation are core desiderata of machine learning research. Language varies along several axes, most importantly, language instance (e.g. French) and domain (e.g. news). While adapting NLP models to new languages within a single domain, or to new domains within a single language, is widely studied, research in joint adaptation is hampered by the lack of evaluation datasets. This prevents the transfer of NLP systems from well-resourced languages and domains to non-dominant language-domain combinations. To address this gap, we introduce M2QA, a multi-domain multilingual question answering benchmark. M2QA includes 13,500 SQuAD 2.0-style question-answer instances in German, Turkish, and Chinese for the domains of product reviews, news, and creative writing. We use M2QA to explore cross-lingual cross-domain performance of fine-tuned models and state-of-the-art LLMs and investigate modular approaches to domain and language adaptation. We witness 1) considerable performance variations across domain-language combinations within model classes and 2) considerable performance drops between source and target language-domain combinations across all model sizes. We demonstrate that M2QA is far from solved, and new methods to effectively transfer both linguistic and domain-specific information are necessary. We make M2QA publicly available at https://github.com/UKPLab/m2qa.

通过引入多领域多语言问题回答基准M2QA，并利用其探索经过微调的模型、最新最先进的LLM的跨语言跨领域性能，以及研究模块化方法对领域和语言进行适应，我们发现模型类别内在领域-语言组合上具有相当大的性能差异，并且在所有模型规模上源语言-目标语言/领域组合之间存在显著的性能下降。我们证明了M2QA并未得到完全解决，需要新的方法来有效地传递语言和领域特定信息。

M2QA：多领域多语言问答