Deep generative models reproduce complex empirical data but cannot extrapolate to novel environments. An intuitive idea to promote extrapolation capabilities is to enforce the architecture to have the modular structure of a causal graphical model, where one can intervene on each module independently of the others in the graph. We develop a framework to formalize this intuition, using the principle of Independent Causal Mechanisms, and show how over-parameterization of generative neural networks can hinder extrapolation capabilities. Our experiments on the generation of human faces shows successive layers of a generator architecture implement independent mechanisms to some extent, allowing meaningful extrapolations. Finally, we illustrate that independence of mechanisms may be enforced during training to improve extrapolation.

生成模型可以通过实现真实数据生成过程的因果图结构向未被观测环境进行预测，然而，由于因果图结构节点通常未被观测，这导致了模型的超参数化和因果结构的不可识别性，此文章开发了一种理论框架，通过一个弱化的可识别性标准解决这种情况，并展示经典的随机梯度下降可能阻碍模型对未被观测数据的预测能力，建议在训练期间明确地强制执行机制的独立性，使用真实世界数据训练的深度生成模型实验证明这些看法，并展示这些模型的外推能力如何被利用。

生成模型中外推独立机制的理论