When machine learning models are deployed on a test distribution different
from the training distribution, they can perform poorly, but overestimate their
performance. In this work, we aim to better estimate a model's performance
under distribution shift, without supervision. To do so, we use a set of
domain-invariant predictors as a proxy for the unknown, true target labels.
Since the error of the resulting risk estimate depends on the target risk of
the proxy model, we study generalization of domain-invariant representations
and show that the complexity of the latent representation has a significant
influence on the target risk. Empirically, our approach (1) enables self-tuning
of domain adaptation models, and (2) accurately estimates the target error of
given models under distribution shift. Other applications include model
selection, deciding early stopping and error detection.

本篇论文研究机器学习模型在不同测试分布的情况下表现不佳且过度估计它们的表现的问题，并提出一种基于领域不变性预测模型的方法来更好地估计模型在转移学习领域的性能，从而实现了领域自适应和对给定模型在分布转移情况下进行准确的目标误差估计，并可以用于模型选择、决定早期停机和错误检测。