This paper describes our system developed for the SemEval-2024 Task 1:
Semantic Textual Relatedness. The challenge is focused on automatically
detecting the degree of relatedness between pairs of sentences for 14 languages
including both high and low-resource Asian and African languages. Our team
participated in two subtasks consisting of Track A: supervised and Track B:
unsupervised. This paper focuses on a BERT-based contrastive learning and
similarity metric based approach primarily for the supervised track while
exploring autoencoders for the unsupervised track. It also aims on the creation
of a bigram relatedness corpus using negative sampling strategy, thereby
producing refined word embeddings.

该研究描述了我们为 SemEval-2024 任务 1 开发的系统：语义文本相关性。该挑战主要关注在包括亚洲和非洲的高资源和低资源语言在内的 14 种语言中自动检测句子对相关性的程度。我们团队参与了两个子任务，包括监督式轨道和非监督式轨道。本文主要关注基于 BERT 的对比学习和基于相似度度量的方法，用于监督式轨道，并探索无监督式轨道中的自动编码器。同时，该研究还旨在使用负采样策略创建一个双词相关性语料库，从而生成精炼的词嵌入。

IITK 参加 SemEval-2024 任务 1：对比学习和自编码器用于多语言文本的语义文本相关性

IITK at SemEval-2024 Task 1: Contrastive Learning and Autoencoders for  Semantic Textual Relatedness in Multilingual Texts

This paper presents our system developed for the SemEval-2024 Task 1:
Semantic Textual Relatedness for African and Asian Languages. The shared task
aims at measuring the semantic textual relatedness between pairs of sentences,
with a focus on a range of under-represented languages. In this work, we
propose using machine translation for data augmentation to address the
low-resource challenge of limited training data. Moreover, we apply
task-adaptive pre-training on unlabeled task data to bridge the gap between
pre-training and task adaptation. For model training, we investigate both full
fine-tuning and adapter-based tuning, and adopt the adapter framework for
effective zero-shot cross-lingual transfer. We achieve competitive results in
the shared task: our system performs the best among all ranked teams in both
subtask A (supervised learning) and subtask C (cross-lingual transfer).

本文提出了在 SemEval-2024 Task 1 中开发的系统：用于非洲和亚洲语言的语义文本相关性。该共享任务旨在测量句子对之间的语义文本相关性，重点关注一系列少数语言。我们在这项工作中提出了使用机器翻译进行数据增强来解决有限训练数据的低资源挑战。此外，我们对未标记任务数据进行任务自适应预训练，以弥合预训练和任务适应之间的差距。对于模型训练，我们研究了完全微调和基于适配器的微调，并采用适配器框架实现了有效的零次跨语言迁移。在共享任务中，我们取得了具有竞争力的结果：我们的系统在子任务 A（监督学习）和子任务 C（跨语言迁移）中排名最高。