This paper describes the results of SemEval 2023 task 7 -- Multi-Evidence
Natural Language Inference for Clinical Trial Data (NLI4CT) -- consisting of 2
tasks, a Natural Language Inference (NLI) task, and an evidence selection task
on clinical trial data. The proposed challenges require multi-hop biomedical
and numerical reasoning, which are of significant importance to the development
of systems capable of large-scale interpretation and retrieval of medical
evidence, to provide personalized evidence-based care.
Task 1, the entailment task, received 643 submissions from 40 participants,
and Task 2, the evidence selection task, received 364 submissions from 23
participants. The tasks are challenging, with the majority of submitted systems
failing to significantly outperform the majority class baseline on the
entailment task, and we observe significantly better performance on the
evidence selection task than on the entailment task. Increasing the number of
model parameters leads to a direct increase in performance, far more
significant than the effect of biomedical pre-training. Future works could
explore the limitations of large models for generalization and numerical
inference, and investigate methods to augment clinical datasets to allow for
more rigorous testing and to facilitate fine-tuning.
We envisage that the dataset, models, and results of this task will be useful
to the biomedical NLI and evidence retrieval communities. The dataset,
competition leaderboard, and website are publicly available.

这篇论文介绍了 SemEval 2023 任务 7 的结果 -- 临床试验数据的多证据自然语言推断（NLI4CT），包括两个任务：自然语言推断任务和临床试验数据的证据选择任务。

SemEval-2023 任务 7：面向临床试验数据的多证据自然语言推理

SemEval-2023 Task 7: Multi-Evidence Natural Language Inference for  Clinical Trial Data

We introduce RoMQA, the first benchmark for robust, multi-evidence,
multi-answer question answering (QA). RoMQA contains clusters of questions that
are derived from related constraints mined from the Wikidata knowledge graph.
RoMQA evaluates robustness of QA models to varying constraints by measuring
worst-case performance within each question cluster. Compared to prior QA
datasets, RoMQA has more human-written questions that require reasoning over
more evidence text and have, on average, many more correct answers. In
addition, human annotators rate RoMQA questions as more natural or likely to be
asked by people. We evaluate state-of-the-art large language models in
zero-shot, few-shot, and fine-tuning settings, and find that RoMQA is
challenging: zero-shot and few-shot models perform similarly to naive
baselines, while supervised retrieval methods perform well below gold evidence
upper bounds. Moreover, existing models are not robust to variations in
question constraints, but can be made more robust by tuning on clusters of
related questions. Our results show that RoMQA is a challenging benchmark for
large language models, and provides a quantifiable test to build more robust QA
methods.

RoMQA 是第一个鲁棒、多证据、多答案问答基准测试，它基于 Wikidata 知识图谱的相关限制生成问题集群，并通过测量每个问题集群中的最坏性能来评估 QA 模型对各种限制的鲁棒性。与先前的 QA 数据集相比，RoMQA 具有更多需要对更多证据文本进行推理的人类编写问题，并且平均有更多正确答案。此外，人类注释员评价 RoMQA 问题更自然或更有可能被人们问到。