Covariate shift in the test data can significantly downgrade both the
accuracy and the fairness performance of the model. Ensuring fairness across
different sensitive groups in such settings is of paramount importance due to
societal implications like criminal justice. We operate under the unsupervised
regime where only a small set of unlabeled test samples along with a labeled
training set is available. Towards this problem, we make three contributions.
First is a novel composite weighted entropy based objective for prediction
accuracy which is optimized along with a representation matching loss for
fairness. We experimentally verify that optimizing with our loss formulation
outperforms a number of state-of-the-art baselines in the pareto sense with
respect to the fairness-accuracy tradeoff on several standard datasets. Our
second contribution is a new setting we term Asymmetric Covariate Shift that,
to the best of our knowledge, has not been studied before. Asymmetric covariate
shift occurs when distribution of covariates of one group shifts significantly
compared to the other groups and this happens when a dominant group is
over-represented. While this setting is extremely challenging for current
baselines, We show that our proposed method significantly outperforms them. Our
third contribution is theoretical, where we show that our weighted entropy term
along with prediction loss on the training set approximates test loss under
covariate shift. Empirically and through formal sample complexity bounds, we
show that this approximation to the unseen test loss does not depend on
importance sampling variance which affects many other baselines.

在测试数据中的协变量偏移可以显著降低模型的准确性和公平性表现。我们提出了一种基于加权熵的预测准确性目标和表示匹配损失的新型组合目标函数，通过实验证明了我们的损失函数优化在公平性与准确性的权衡上优于其他基线方法。我们还提出了一种称为不对称协变量偏移的新颖设置，并展示了我们的方法在这种设置下的显著优势。最后，我们理论上证明了在训练集上的加权熵项和预测损失可以近似于协变量偏移时的测试损失。我们通过实验证明了这种对未见测试损失的近似不依赖于影响其他基线方法的重要性采样方差。