Code-mixed discourse combines multiple languages in a single text. It is
commonly used in informal discourse in countries with several official
languages, but also in many other countries in combination with English or
neighboring languages. As recently large language models have dominated most
natural language processing tasks, we investigated their performance in
code-mixed settings for relevant tasks. We first created four new bilingual
pre-trained masked language models for English-Hindi and English-Slovene
languages, specifically aimed to support informal language. Then we performed
an evaluation of monolingual, bilingual, few-lingual, and massively
multilingual models on several languages, using two tasks that frequently
contain code-mixed text, in particular, sentiment analysis and offensive
language detection in social media texts. The results show that the most
successful classifiers are fine-tuned bilingual models and multilingual models,
specialized for social media texts, followed by non-specialized massively
multilingual and monolingual models, while huge generative models are not
competitive. For our affective problems, the models mostly perform slightly
better on code-mixed data compared to non-code-mixed data.

研究发现，针对社交媒体文本中的混合编码文本进行专门设计的双语模型和多语模型表现最佳，而庞大生成模型则不具有竞争力。对于情感分析和辱骂语言检测等任务，这些模型在混合编码数据上的表现稍微优于非混合编码数据。