We introduce KazQAD -- a Kazakh open-domain question answering (ODQA) dataset
-- that can be used in both reading comprehension and full ODQA settings, as
well as for information retrieval experiments. KazQAD contains just under 6,000
unique questions with extracted short answers and nearly 12,000 passage-level
relevance judgements. We use a combination of machine translation, Wikipedia
search, and in-house manual annotation to ensure annotation efficiency and data
quality. The questions come from two sources: translated items from the Natural
Questions (NQ) dataset (only for training) and the original Kazakh Unified
National Testing (UNT) exam (for development and testing). The accompanying
text corpus contains more than 800,000 passages from the Kazakh Wikipedia. As a
supplementary dataset, we release around 61,000 question-passage-answer triples
from the NQ dataset that have been machine-translated into Kazakh. We develop
baseline retrievers and readers that achieve reasonable scores in retrieval
(NDCG@10 = 0.389 MRR = 0.382), reading comprehension (EM = 38.5 F1 = 54.2), and
full ODQA (EM = 17.8 F1 = 28.7) settings. Nevertheless, these results are
substantially lower than state-of-the-art results for English QA collections,
and we think that there should still be ample room for improvement. We also
show that the current OpenAI's ChatGPTv3.5 is not able to answer KazQAD test
questions in the closed-book setting with acceptable quality. The dataset is
freely available under the Creative Commons licence (CC BY-SA) at
this https URL

我们介绍了 KazQAD 这个哈萨克斯坦的开放域问题回答数据集，可用于阅读理解和完整的 ODQA 设置，以及信息检索实验。该数据集包含近 6,000 个独特问题、提取的简短答案和将近 12,000 个段落级相关性判断。我们使用机器翻译、维基百科搜索和内部手动注释的组合来确保注释效率和数据质量。问题来自两个来源：从自然问题（NQ）数据集翻译而来的项目（仅用于训练）和哈萨克统一国家考试（UNT）的原始考题（用于开发和测试）。相关的文本语料库包含来自哈萨克斯坦维基百科的超过 800,000 个段落。作为补充数据集，我们发布了大约 61,000 个问题 - 段落 - 答案三元组，这些三元组已经被机器翻译成了哈萨克语。我们开发了基准检索器和阅读器，在检索（NDCG@10 = 0.389 MRR = 0.382）、阅读理解（EM = 38.5 F1 = 54.2）和完整的 ODQA（EM = 17.8 F1 = 28.7）设置下取得了合理的分数。然而，与英文 QA 集合的最新结果相比，这些结果要低得多，我们认为仍然有很大的改进空间。我们还表明目前的 OpenAI 的 ChatGPTv3.5 无法以可接受的质量回答 KazQAD 的测试问题（闭卷设置）。该数据集在创作公共许可证（CC BY-SA）下免费提供，链接为 https URL。