Adversarial evaluations of language models typically focus on English alone.
In this paper, we performed a multilingual evaluation of Named Entity
Recognition (NER) in terms of its robustness to small perturbations in the
input. Our results showed the NER models we explored across three languages
(English, German and Hindi) are not very robust to such changes, as indicated
by the fluctuations in the overall F1 score as well as in a more fine-grained
evaluation. With that knowledge, we further explored whether it is possible to
improve the existing NER models using a part of the generated adversarial data
sets as augmented training data to train a new NER model or as fine-tuning data
to adapt an existing NER model. Our results showed that both these approaches
improve performance on the original as well as adversarial test sets. While
there is no significant difference between the two approaches for English,
re-training is significantly better than fine-tuning for German and Hindi.

本文在多种语言中对命名实体识别 (Named Entity Recognition, NER) 进行了对抗性评估，并尝试使用对抗性数据集作为训练数据或微调数据以提高 NER 模型的性能。结果表明，这两种方法均可提高英语、德语和印地语的 NER 性能，但对于德语和印地语而言，重新训练的效果更好。