When designing evidence-based policies and programs, decision-makers must
distill key information from a vast and rapidly growing literature base.
Identifying relevant literature from raw search results is time and resource
intensive, and is often done by manual screening. In this study, we develop an
AI agent based on a bidirectional encoder representations from transformers
(BERT) model and incorporate it into a human team designing an evidence
synthesis product for global development. We explore the effectiveness of the
human-AI hybrid team in accelerating the evidence synthesis process. To further
improve team efficiency, we enhance the human-AI hybrid team through active
learning (AL). Specifically, we explore different sampling strategies,
including random sampling, least confidence (LC) sampling, and highest priority
(HP) sampling, to study their influence on the collaborative screening process.
Results show that incorporating the BERT-based AI agent into the human team can
reduce the human screening effort by 68.5% compared to the case of no AI
assistance and by 16.8% compared to the case of using a support vector machine
(SVM)-based AI agent for identifying 80% of all relevant documents. When we
apply the HP sampling strategy for AL, the human screening effort can be
reduced even more: by 78.3% for identifying 80% of all relevant documents
compared to no AI assistance. We apply the AL-enhanced human-AI hybrid teaming
workflow in the design process of three evidence gap maps (EGMs) for USAID and
find it to be highly effective. These findings demonstrate how AI can
accelerate the development of evidence synthesis products and promote timely
evidence-based decision making in global development in a human-AI hybrid
teaming context.

本研究介绍了一种利用 BERT 模型开发 AI 代理程序，结合人类团队设计全球发展证据综合产品的方法，探讨了活性学习策略的影响，结果表明，将 BERT-based AI 代理程序纳入人类团队可以降低人类筛选工作量 68.5％，而采用高优先级 (HP) 策略的活性学习方法可以进一步减少 78.3％。

ADVISE: 全球发展证据综述的 AI 加速设计

ADVISE: AI-accelerated Design of Evidence Synthesis for Global  Development

Climate change is threatening human health in unprecedented orders and many
ways. These threats are expected to grow unless effective and evidence-based
policies are developed and acted upon to minimize or eliminate them. Attaining
such a task requires the highest degree of the flow of knowledge from science
into policy. The multidisciplinary, location-specific, and vastness of
published science makes it challenging to keep track of novel work in this
area, as well as making the traditional knowledge synthesis methods inefficient
in infusing science into policy. To this end, we consider developing multiple
domain-specific language models (LMs) with different variations from Climate-
and Health-related information, which can serve as a foundational step toward
capturing available knowledge to enable solving different tasks, such as
detecting similarities between climate- and health-related concepts,
fact-checking, relation extraction, evidence of health effects to policy text
generation, and more. To our knowledge, this is the first work that proposes
developing multiple domain-specific language models for the considered domains.
We will make the developed models, resources, and codebase available for the
researchers.

本文提出了在气候和健康领域开发多个领域特定的语言模型，通过捕捉可用的知识来解决各种任务，从而消除气候变化对人类健康的影响。这是第一篇在这些领域考虑开发多个领域特定语言模型的工作，并将所开发的模型、资源和代码库提供给研究人员。