Text classification, an integral task in natural language processing,
involves the automatic categorization of text into predefined classes. Creating
supervised labeled datasets for low-resource languages poses a considerable
challenge. Unlocking the language potential of low-resource languages requires
robust datasets with supervised labels. However, such datasets are scarce, and
the label space is often limited. In our pursuit to address this gap, we aim to
optimize existing labels/datasets in different languages. This research
proposes a novel perspective on Universal Cross-Lingual Text Classification,
leveraging a unified model across languages. Our approach involves blending
supervised data from different languages during training to create a universal
model. The supervised data for a target classification task might come from
different languages covering different labels. The primary goal is to enhance
label and language coverage, aiming for a label set that represents a union of
labels from various languages. We propose the usage of a strong multilingual
SBERT as our base model, making our novel training strategy feasible. This
strategy contributes to the adaptability and effectiveness of the model in
cross-lingual language transfer scenarios, where it can categorize text in
languages not encountered during training. Thus, the paper delves into the
intricacies of cross-lingual text classification, with a particular focus on
its application for low-resource languages, exploring methodologies and
implications for the development of a robust and adaptable universal
cross-lingual model.

通过优化现有标签 / 数据集，将不同语言的标签数据与训练相结合，以创建一个通用模型，从而增强标签和语言范围，实现跨语言的文本分类。

通用跨语言文本分类

Universal Cross-Lingual Text Classification

The multilingual Sentence-BERT (SBERT) models map different languages to
common representation space and are useful for cross-language similarity and
mining tasks. We propose a simple yet effective approach to convert vanilla
multilingual BERT models into multilingual sentence BERT models using synthetic
corpus. We simply aggregate translated NLI or STS datasets of the low-resource
target languages together and perform SBERT-like fine-tuning of the vanilla
multilingual BERT model. We show that multilingual BERT models are inherent
cross-lingual learners and this simple baseline fine-tuning approach without
explicit cross-lingual training yields exceptional cross-lingual properties. We
show the efficacy of our approach on 10 major Indic languages and also show the
applicability of our approach to non-Indic languages German and French. Using
this approach, we further present L3Cube-IndicSBERT, the first multilingual
sentence representation model specifically for Indian languages Hindi, Marathi,
Kannada, Telugu, Malayalam, Tamil, Gujarati, Odia, Bengali, and Punjabi. The
IndicSBERT exhibits strong cross-lingual capabilities and performs
significantly better than alternatives like LaBSE, LASER, and
paraphrase-multilingual-mpnet-base-v2 on Indic cross-lingual and monolingual
sentence similarity tasks. We also release monolingual SBERT models for each of
the languages and show that IndicSBERT performs competitively with its
monolingual counterparts. These models have been evaluated using embedding
similarity scores and classification accuracy.

该论文提出了一种用于转换 vanilla multilingual BERT 模型成多语言句子 BERT 模型的简单有效方法，该方法使用合成语料库聚合并细调 vanilla multilingual BERT 模型的翻译 NLI 或 STS 数据集，展示了多语言 BERT 模型的跨语言学习能力，该方法在 10 种主要的印度语言以及德语和法语这些非印欧语系的语言上均取得了很好的效果，进而实现了首个针对印度语言 Hindi、Marathi、Kannada、Telugu、Malayalam、Tamil、Gujarati、Odia、Bengali 和 Punjabi 的多语言句子表示模型。