The increasing rate at which scientific knowledge is discovered and health
claims shared online has highlighted the importance of developing efficient
fact-checking systems for scientific claims. The usual setting for this task in
the literature assumes that the documents containing the evidence for claims
are already provided and annotated or contained in a limited corpus. This
renders the systems unrealistic for real-world settings where knowledge sources
with potentially millions of documents need to be queried to find relevant
evidence. In this paper, we perform an array of experiments to test the
performance of open-domain claim verification systems. We test the final
verdict prediction of systems on four datasets of biomedical and health claims
in different settings. While keeping the pipeline's evidence selection and
verdict prediction parts constant, document retrieval is performed over three
common knowledge sources (PubMed, Wikipedia, Google) and using two different
information retrieval techniques. We show that PubMed works better with
specialized biomedical claims, while Wikipedia is more suited for everyday
health concerns. Likewise, BM25 excels in retrieval precision, while semantic
search in recall of relevant evidence. We discuss the results, outline frequent
retrieval patterns and challenges, and provide promising future directions.

本研究通过一系列实验测试了开放领域的声明验证系统的性能，在不同设置下使用三个常见知识来源（PubMed、Wikipedia、Google）和两种不同的信息检索技术，对四个生物医学和健康声明的最终预测进行了测试。结果显示，对于专门的生物医学声明，PubMed 表现更好，而对于日常健康问题，Wikipedia 更适合；而在检索精度方面，BM25 表现出色，而在相关证据的回收方面，语义搜索更具优势。在讨论结果、概述频繁的检索模式和挑战，并提供有前途的未来方向。