Recently proposed systems for open-domain question answering (OpenQA) require
large amounts of training data to achieve state-of-the-art performance.
However, data annotation is known to be time-consuming and therefore expensive
to acquire. As a result, the appropriate datasets are available only for a
handful of languages (mainly English and Chinese). In this work, we introduce
and publicly release PolQA, the first Polish dataset for OpenQA. It consists of
7,000 questions, 87,525 manually labeled evidence passages, and a corpus of
over 7,097,322 candidate passages. Each question is classified according to its
formulation, type, as well as entity type of the answer. This resource allows
us to evaluate the impact of different annotation choices on the performance of
the QA system and propose an efficient annotation strategy that increases the
passage retrieval performance by 10.55 p.p. while reducing the annotation cost
by 82%.

本文介绍并公开发布了 PolQA，这是第一个用于 OpenQA 的波兰语数据集。该数据集包含 7,000 个问题，87,525 个手动标注的证据段落和超过 7,097,322 个候选段落。该资源允许我们评估不同注释选择对 QA 系统性能的影响，并提出一种有效的注释策略，使召回率提高了 10.55 p.p. 并同时将注释成本降低了 82％。