Prediction becomes more challenging with missing covariates. What method is
chosen to handle missingness can greatly affect how models perform. In many
real-world problems, the best prediction performance is achieved by models that
can leverage the informative nature of a value being missing. Yet, the reasons
why a covariate goes missing can change once a model is deployed in practice.
If such a missingness shift occurs, the conditional probability of a value
being missing differs in the target data. Prediction performance in the source
data may no longer be a good selection criterion, and approaches that do not
rely on informative missingness may be preferable. However, we show that the
Bayes predictor remains unchanged by ignorable shifts for which the probability
of missingness only depends on observed data. Any consistent estimator of the
Bayes predictor may therefore result in robust prediction under those
conditions, although we show empirically that different methods appear robust
to different types of shifts. If the missingness shift is non-ignorable, the
Bayes predictor may change due to the shift. While neither approach recovers
the Bayes predictor in this case, we found empirically that disregarding
missingness was most beneficial when it was highly informative.

预测表现受到缺失协变量的影响，处理缺失数据的方法选择会极大地影响模型性能。研究发现，具有信息性缺失值的模型可以实现最佳的预测性能，但实际部署后缺失值产生的原因可能会发生变化，此时基于缺失的条件概率在目标数据中会有所不同。因此，如果缺失值的产生是不可忽略的，Bayes 预测器可能会因为缺失值的变化而有所改变。然而，我们的实证研究发现，在高度信息化的情况下，忽略缺失值是最有益的。

缺失偏移下的稳健预测

Robust prediction under missingness shifts

Machine learning systems often experience a distribution shift between
training and testing. In this paper, we introduce a simple variational
objective whose optima are exactly the set of all representations on which risk
minimizers are guaranteed to be robust to any distribution shift that preserves
the Bayes predictor, e.g., covariate shifts. Our objective has two components.
First, a representation must remain discriminative for the task, i.e., some
predictor must be able to simultaneously minimize the source and target risk.
Second, the representation's marginal support needs to be the same across
source and target. We make this practical by designing self-supervised
objectives that only use unlabelled data and augmentations to train robust
representations. Our objectives give insights into the robustness of CLIP, and
further improve CLIP's representations to achieve SOTA results on DomainBed.

介绍了一种简单的变分目标，其极值恰好是所有表示集合，其风险最小化器保证对任何保留贝叶斯预测者的分布转移都是强健的，例如，协变量转移。该方法包含了两个方面，第一个方面是表示必须保持判别，即某些预测者必须能够同时使源风险和目标风险最小化；第二个方面是表示的边际支持在源和目标之间需要相同。通过设计自监督目标来训练稳健的表示，从而使此方法成为实际可行。本文还研究了 CLIP 的鲁棒性并改进了其表示，以在 DomainBed 上实现 SOTA 结果。