The gold standard for causal model evaluation involves comparing model
predictions with true effects estimated from randomized controlled trials
(RCT). However, RCTs are not always feasible or ethical to perform. In
contrast, conditionally randomized experiments based on inverse probability
weighting (IPW) offer a more realistic approach but may suffer from high
estimation variance. To tackle this challenge and enhance causal model
evaluation in real-world conditional randomization settings, we introduce a
novel low-variance estimator for causal error, dubbed as the pairs estimator.
By applying the same IPW estimator to both the model and true experimental
effects, our estimator effectively cancels out the variance due to IPW and
achieves a smaller asymptotic variance. Empirical studies demonstrate the
improved of our estimator, highlighting its potential on achieving near-RCT
performance. Our method offers a simple yet powerful solution to evaluate
causal inference models in conditional randomization settings without
complicated modification of the IPW estimator itself, paving the way for more
robust and reliable model assessments.

我们引入了一种新的低方差评估因果误差的估计方法，称为 pairs estimator。通过将同样的 IPW 估计器应用于模型和真实实验效果，我们的估计器有效地消除了由于 IPW 引起的方差，并实现了较小的渐近方差。经验证明，我们的估计器的性能得到了改进，几乎能达到 RCT 的性能水平。我们的方法提供了一种简单而强大的解决方案，用于在条件随机化设置中评估因果推断模型，无需对 IPW 估计器本身进行复杂修改，为更健壮可靠的模型评估铺平了道路。