Code-switching is a prevalent linguistic phenomenon in which multilingual
individuals seamlessly alternate between languages. Despite its widespread use
online and recent research trends in this area, research in code-switching
presents unique challenges, primarily stemming from the scarcity of labelled
data and available resources. In this study we investigate how pre-trained
Language Models handle code-switched text in three dimensions: a) the ability
of PLMs to detect code-switched text, b) variations in the structural
information that PLMs utilise to capture code-switched text, and c) the
consistency of semantic information representation in code-switched text. To
conduct a systematic and controlled evaluation of the language models in
question, we create a novel dataset of well-formed naturalistic code-switched
text along with parallel translations into the source languages. Our findings
reveal that pre-trained language models are effective in generalising to
code-switched text, shedding light on the abilities of these models to
generalise representations to CS corpora. We release all our code and data
including the novel corpus at this https URL

本研究通过对预训练语言模型处理混合语言文本的能力、模型捕捉混合语言文本的结构信息的变化性以及语义信息表达的一致性的研究，揭示了预训练语言模型在泛化到混合语言文本上的有效性，从而为这些模型在处理混合语言资源方面的能力提供了洞察。