Incorporating unanswerable questions into EHR QA systems is crucial for
testing the trustworthiness of a system, as providing non-existent responses
can mislead doctors in their diagnoses. The EHRSQL dataset stands out as a
promising benchmark because it is the only dataset that incorporates
unanswerable questions in the EHR QA system alongside practical questions.
However, in this work, we identify a data bias in these unanswerable questions;
they can often be discerned simply by filtering with specific N-gram patterns.
Such biases jeopardize the authenticity and reliability of QA system
evaluations. To tackle this problem, we propose a simple debiasing method of
adjusting the split between the validation and test sets to neutralize the
undue influence of N-gram filtering. By experimenting on the MIMIC-III dataset,
we demonstrate both the existing data bias in EHRSQL and the effectiveness of
our data split strategy in mitigating this bias.

为了测试系统的可信度，将无法回答的问题纳入电子病历问答系统是至关重要的，因为提供不存在的答案可能会误导医生的诊断。然而，我们的工作中发现了 EHR QA 系统中这些无法回答问题存在的数据偏见，它们往往可以通过特定的 N-gram 模式过滤来辨别。这样的偏见危及 QA 系统评估的真实性和可靠性。为解决此问题，我们提出了一种简单的去偏方法，通过调整验证集和测试集之间的分割来中和 N-gram 过滤的不适当影响。通过对 MIMIC-III 数据集的实验，我们展示了 EHRSQL 的现有数据偏见以及我们的数据划分策略在减轻这种偏见方面的有效性。