With the growing access to administrative health databases, retrospective
studies have become crucial evidence for medical treatments. Yet,
non-randomized studies frequently face selection biases, requiring mitigation
strategies. Propensity score matching (PSM) addresses these biases by selecting
comparable populations, allowing for analysis without further methodological
constraints. However, PSM has several drawbacks. Different matching methods can
produce significantly different Average Treatment Effects (ATE) for the same
task, even when meeting all validation criteria. To prevent cherry-picking the
best method, public authorities must involve field experts and engage in
extensive discussions with researchers.
To address this issue, we introduce a novel metric, A2A, to reduce the number
of valid matches. A2A constructs artificial matching tasks that mirror the
original ones but with known outcomes, assessing each matching method's
performance comprehensively from propensity estimation to ATE estimation. When
combined with Standardized Mean Difference, A2A enhances the precision of model
selection, resulting in a reduction of up to 50% in ATE estimation errors
across synthetic tasks and up to 90% in predicted ATE variability across both
synthetic and real-world datasets. To our knowledge, A2A is the first metric
capable of evaluating outcome correction accuracy using covariates not involved
in selection.
Computing A2A requires solving hundreds of PSMs, we therefore automate all
manual steps of the PSM pipeline. We integrate PSM methods from Python and R,
our automated pipeline, a new metric, and reproducible experiments into
popmatch, our new Python package, to enhance reproducibility and accessibility
to bias correction methods.

通过引入新的度量标准 A2A 来减少有效匹配数量，A2A 能够综合评估每种匹配方法的性能，从倾向估计到平均处理效应估计，提高模型选择的精确性，并在合成任务中减少高达 50% 的 ATE 估计误差，在合成和实际数据集中降低高达 90% 的预测 ATE 变异性。

量化偏差修正标准对治疗结果的影响的改进

Improving Bias Correction Standards by Quantifying its Effects on  Treatment Outcomes

Identifying patient cohorts is fundamental to numerous healthcare tasks,
including clinical trial recruitment and retrospective studies. Current cohort
retrieval methods in healthcare organizations rely on automated queries of
structured data combined with manual curation, which are time-consuming,
labor-intensive, and often yield low-quality results. Recent advancements in
large language models (LLMs) and information retrieval (IR) offer promising
avenues to revolutionize these systems. Major challenges include managing
extensive eligibility criteria and handling the longitudinal nature of
unstructured Electronic Medical Records (EMRs) while ensuring that the solution
remains cost-effective for real-world application. This paper introduces a new
task, Automatic Cohort Retrieval (ACR), and evaluates the performance of LLMs
and commercial, domain-specific neuro-symbolic approaches. We provide a
benchmark task, a query dataset, an EMR dataset, and an evaluation framework.
Our findings underscore the necessity for efficient, high-quality ACR systems
capable of longitudinal reasoning across extensive patient databases.

自动检索病例组是许多医疗任务的基础，本文引入了一种新的任务 —— 自动检索病例组，并评估了大型语言模型和领域特定的神经符号方法的性能。

ACR：自动队友检索基准

ACR: A Benchmark for Automatic Cohort Retrieval

Toxic misinformation campaigns have caused significant societal harm, e.g.,
affecting elections and COVID-19 information awareness. Unfortunately, despite
successes of (gold standard) retrospective studies of misinformation that
confirmed their harmful effects after the fact, they arrive too late for timely
intervention and reduction of such harm. By design, misinformation evades
retrospective classifiers by exploiting two properties we call new-normal: (1)
never-seen-before novelty that cause inescapable generalization challenges for
previous classifiers, and (2) massive but short campaigns that end before they
can be manually annotated for new classifier training. To tackle these
challenges, we propose UFIT, which combines two techniques: semantic masking of
strong signal keywords to reduce overfitting, and intra-proxy smoothness
regularization of high-density regions in the latent space to improve
reliability and maintain accuracy. Evaluation of UFIT on public new-normal
misinformation data shows over 30% improvement over existing approaches on
future (and unseen) campaigns. To the best of our knowledge, UFIT is the first
successful effort to achieve such high level of generalization on new-normal
misinformation data with minimal concession (1 to 5%) of accuracy compared to
oracles trained with full knowledge of all campaigns.

研究提出了 UFIT 技术，通过语义遮蔽和内部代理平滑正则化技术来解决 “新常态” 的虚假信息的泛化问题，达到了 30% 以上的改进效果，并在保持最佳精度的情况下实现了最高级别的泛化。