Distribution shifts are common in real-world datasets and can affect the
performance and reliability of deep learning models. In this paper, we study
two types of distribution shifts: diversity shifts, which occur when test
samples exhibit patterns unseen during training, and correlation shifts, which
occur when test data present a different correlation between seen invariant and
spurious features. We propose an integrated protocol to analyze both types of
shifts using datasets where they co-exist in a controllable manner. Finally, we
apply our approach to a real-world classification problem of skin cancer
analysis, using out-of-distribution datasets and specialized bias annotations.
Our protocol reveals three findings: 1) Models learn and propagate correlation
shifts even with low-bias training; this poses a risk of accumulating and
combining unaccountable weak biases; 2) Models learn robust features in high-
and low-bias scenarios but use spurious ones if test samples have them; this
suggests that spurious correlations do not impair the learning of robust
features; 3) Diversity shift can reduce the reliance on spurious correlations;
this is counter intuitive since we expect biased models to depend more on
biases when invariant features are missing. Our work has implications for
distribution shift research and practice, providing new insights into how
models learn and rely on spurious correlations under different types of shifts.

本文针对分布转移对深度学习模型性能和可靠性的影响，研究了两种类型的分布转移：多样性转移和相关性转移，并提出了一种综合协议来分析这两种转移。应用我们的方法到一个真实的皮肤癌分类问题，最终有三个发现：模型在低偏差情况下仍学习和传播与转移相关的内容；模型学习了强健的特征但仍使用其他的特征；多样性转移可以降低偏差模型对于与转移相关的特征的依赖性。