Evaluating the veracity of everyday claims is time consuming and in some
cases requires domain expertise. We empirically demonstrate that the commonly
used fact checking pipeline, known as the retriever-reader, suffers from
performance deterioration when it is trained on the labeled data from one
domain and used in another domain. Afterwards, we delve into each component of
the pipeline and propose novel algorithms to address this problem. We propose
an adversarial algorithm to make the retriever component robust against
distribution shift. Our core idea is to initially train a bi-encoder on the
labeled source data, and then, to adversarially train two separate document and
claim encoders using unlabeled target data. We then focus on the reader
component and propose to train it such that it is insensitive towards the order
of claims and evidence documents. Our empirical evaluations support the
hypothesis that such a reader shows a higher robustness against distribution
shift. To our knowledge, there is no publicly available multi-topic fact
checking dataset. Thus, we propose a simple automatic method to re-purpose two
well-known fact checking datasets. We then construct eight fact checking
scenarios from these datasets, and compare our model to a set of strong
baseline models, including recent domain adaptation models that use GPT4 for
generating synthetic data.

我们提出了针对事实检查领域的常见问题的改进算法，包括对传统的 retriever-reader 模型的性能下降问题的改进，以及对 reader 组件的改进，通过对 claims 和 evidence documents 的无序性进行训练，从而提高对分布偏移的鲁棒性。此外，我们还提出了一种自动方法来构建多主题事实检查数据集，并将我们的模型与一组强大的基准模型进行比较。

训练集以外的事实核查

Fact Checking Beyond Training Set

Open-domain Question Answering (OpenQA) is an important task in Natural
Language Processing (NLP), which aims to answer a question in the form of
natural language based on large-scale unstructured documents. Recently, there
has been a surge in the amount of research literature on OpenQA, particularly
on techniques that integrate with neural Machine Reading Comprehension (MRC).
While these research works have advanced performance to new heights on
benchmark datasets, they have been rarely covered in existing surveys on QA
systems. In this work, we review the latest research trends in OpenQA, with
particular attention to systems that incorporate neural MRC techniques.
Specifically, we begin with revisiting the origin and development of OpenQA
systems. We then introduce modern OpenQA architecture named "Retriever-Reader"
and analyze the various systems that follow this architecture as well as the
specific techniques adopted in each of the components. We then discuss key
challenges to developing OpenQA systems and offer an analysis of benchmarks
that are commonly used. We hope our work would enable researchers to be
informed of the recent advancement and also the open challenges in OpenQA
research, so as to stimulate further progress in this field.

本文综述了 OpenQA 领域的最新研究趋势，重点关注采用神经 MRC 技术的系统，介绍了现代 OpenQA 体系结构 Retriever-Reader 及采用该体系结构的各种系统，讨论了开发 OpenQA 系统面临的关键挑战，并对常用的基准进行了分析。