Most existing work on adversarial data generation focuses on English. For
example, PAWS (Paraphrase Adversaries from Word Scrambling) consists of
challenging English paraphrase identification pairs from Wikipedia and Quora.
We remedy this gap with PAWS-X, a new dataset of 23,659 human translated PAWS
evaluation pairs in six typologically distinct languages: French, Spanish,
German, Chinese, Japanese, and Korean. We provide baseline numbers for three
models with different capacity to capture non-local context and sentence
structure, and using different multilingual training and evaluation regimes.
Multilingual BERT fine-tuned on PAWS English plus machine-translated data
performs the best, with a range of 83.1-90.8 accuracy across the non-English
languages and an average accuracy gain of 23% over the next best model. PAWS-X
shows the effectiveness of deep, multilingual pre-training while also leaving
considerable headroom as a new challenge to drive multilingual research that
better captures structure and contextual information.

PAWS-X 是一个新的数据集，由六种不同类型的语言中的 23,659 个人类翻译评估成对组成，并提供了三个不同容量模型的基线数字，这些模型具有捕捉非本地上下文和句子结构的不同能力，并使用不同的多语种训练和评估模式。