Code-switching is the use of more than one language in the same conversation
or utterance. Recently, multilingual contextual embedding models, trained on
multiple monolingual corpora, have shown promising results on cross-lingual and
multilingual tasks. We present an evaluation benchmark, GLUECoS, for
code-switched languages, that spans several NLP tasks in English-Hindi and
English-Spanish. Specifically, our evaluation benchmark includes Language
Identification from text, POS tagging, Named Entity Recognition, Sentiment
Analysis, Question Answering and a new task for code-switching, Natural
Language Inference. We present results on all these tasks using cross-lingual
word embedding models and multilingual models. In addition, we fine-tune
multilingual models on artificially generated code-switched data. Although
multilingual models perform significantly better than cross-lingual models, our
results show that in most tasks, across both language pairs, multilingual
models fine-tuned on code-switched data perform best, showing that multilingual
models can be further optimized for code-switching tasks.

在一个名为，GLUECoS, 的评估基准中，使用多语境上下文嵌入模型进行评估，评估基准包括领域从文本中识别、POS 标记、命名实体识别、情感分析、问答和一项新的代码交换任务的多个自然语言处理任务。我们使用跨语言和多语言模型在所有这些任务上展示结果，并在人工生成的代码交换数据上微调多语言模型，结果表明多语言模型可以进一步针对代码交换任务进行优化，即使多语言模型比跨语言模型表现显著好，但我们的结果表明，在大多数任务中，无论是跨语言还是多种语言，微调代码交换数据上的多语言模型效果最佳。