Question-answering systems have revolutionized information retrieval, but
linguistic and cultural boundaries limit their widespread accessibility. This
research endeavors to bridge the gap of the absence of efficient QnA datasets
in low-resource languages by translating the English Question Answering Dataset
(SQuAD) using a robust data curation approach. We introduce MahaSQuAD, the
first-ever full SQuAD dataset for the Indic language Marathi, consisting of
118,516 training, 11,873 validation, and 11,803 test samples. We also present a
gold test set of manually verified 500 examples. Challenges in maintaining
context and handling linguistic nuances are addressed, ensuring accurate
translations. Moreover, as a QnA dataset cannot be simply converted into any
low-resource language using translation, we need a robust method to map the
answer translation to its span in the translated passage. Hence, to address
this challenge, we also present a generic approach for translating SQuAD into
any low-resource language. Thus, we offer a scalable approach to bridge
linguistic and cultural gaps present in low-resource languages, in the realm of
question-answering systems. The datasets and models are shared publicly at
this https URL .

通过使用强大的数据策划方法，我们翻译了英语问答数据集（SQuAD），解决了低资源语言中无高效问答数据集的问题。我们引入了 MahaSQuAD，这是第一个适用于印度马拉地语的完整 SQuAD 数据集，包含 118,516 个训练样本、11,873 个验证样本和 11,803 个测试样本。另外，我们还提供了一个手动验证的黄金测试集，包含 500 个例子。通过解决上下文和语言细微差异的挑战，我们确保了准确的翻译。此外，由于无法简单地将问答数据集直接转换为任何低资源语言，我们需要一种强大的方法将答案翻译映射到译文段落中的相应部分。因此，为了解决这一挑战，我们还提出了一种通用方法，可以将 SQuAD 翻译成任何低资源语言。因此，在问题回答系统领域，我们提供了一种可扩展的方法，弥合了低资源语言中存在的语言和文化差距。数据集和模型已公开共享于此 https URL。