We present MaskLID, a simple, yet effective, code-switching (CS) language
identification (LID) method. MaskLID does not require any training and is
designed to complement current high-performance sentence-level LIDs.
Sentence-level LIDs are classifiers trained on monolingual texts to provide
single labels, typically using a softmax layer to turn scores into
probabilities. However, in cases where a sentence is composed in both L1 and L2
languages, the LID classifier often only returns the dominant label L1. To
address this limitation, MaskLID employs a strategy to mask text features
associated with L1, allowing the LID to classify the text as L2 in the next
round. This method uses the LID itself to identify the features that require
masking and does not rely on any external resource. In this work, we explore
the use of MaskLID for two open-source LIDs (GlotLID and OpenLID), that are
both based on the FastText architecture. Code and demo are available at
this https URL

MaskLID 是一种简单而有效的代码切换语言识别方法，旨在补充当前高性能的句子级别识别器。它通过掩盖与 L1 语言相关的文本特征，使得识别器可以在下一轮中将文本分类为 L2，从而解决仅返回 L1 标签的问题。该方法利用识别器本身来识别需要掩盖的特征，不依赖于任何外部资源。本研究在两种开源识别器 (GlotLID 和 OpenLID) 上探索了 MaskLID 的应用，它们都基于 FastText 架构。