Despite impressive progress in the last decade, it still remains an open challenge to build models that generalize well across multiple tasks and datasets. One path to achieve this is to learn meaningful and compact representations, in which different semantic aspects of data are structurally disentangled. The focus of disentanglement approaches has been on separating independent factors of variation despite the fact that real-world observations are often not structured into meaningful independent causal variables to begin with. In this work we bridge the gap to real-world scenarios by analyzing the behavior of most prominent methods and disentanglement scores on correlated data in a large scale empirical study (including 3900 models). We show that systematically induced correlations in the dataset are being learned and reflected in the latent representations, while widely used disentanglement scores fall short of capturing these latent correlations. Finally, we demonstrate how to disentangle these latent correlations using weak supervision, even if we constrain this supervision to be causally plausible. Our results thus support the argument to learn independent mechanisms rather than independent factors of variations.

本文研究了因果变量在相关数据上的行为并分析了最主要的解缠方法在大规模实证研究（包括4260个模型）中的表现，结果显示数据集中系统性引入的相关性正在被学习并反映在潜在表示中，这对于解缠的下游应用如公平性有重要的影响，同时呈现了如何通过训练期间的弱监督或使用少量标签的预训练模型的后修复来解决这些潜在的相关性。

从相关数据中学习分离表示