The premise of identifiable and causal representation learning is to improve
the current representation learning paradigm in terms of generalizability or
robustness. Despite recent progress in questions of identifiability, more
theoretical results demonstrating concrete advantages of these methods for
downstream tasks are needed. In this paper, we consider the task of
intervention extrapolation: predicting how interventions affect an outcome,
even when those interventions are not observed at training time, and show that
identifiable representations can provide an effective solution to this task
even if the interventions affect the outcome non-linearly. Our setup includes
an outcome Y, observed features X, which are generated as a non-linear
transformation of latent features Z, and exogenous action variables A, which
influence Z. The objective of intervention extrapolation is to predict how
interventions on A that lie outside the training support of A affect Y. Here,
extrapolation becomes possible if the effect of A on Z is linear and the
residual when regressing Z on A has full support. As Z is latent, we combine
the task of intervention extrapolation with identifiable representation
learning, which we call Rep4Ex: we aim to map the observed features X into a
subspace that allows for non-linear extrapolation in A. We show using Wiener's
Tauberian theorem that the hidden representation is identifiable up to an
affine transformation in Z-space, which is sufficient for intervention
extrapolation. The identifiability is characterized by a novel constraint
describing the linearity assumption of A on Z. Based on this insight, we
propose a method that enforces the linear invariance constraint and can be
combined with any type of autoencoder. We validate our theoretical findings
through synthetic experiments and show that our approach succeeds in predicting
the effects of unseen interventions.

本文考虑识别性表示学习和干预外推任务，发现可识别表示能有效预测干预对结果的影响，即使干预呈非线性关系。通过结合干预外推任务和可识别表示学习，我们提出了一种新的方法 Rep4Ex，通过对观察特征进行非线性外推，成功预测未观察干预的效应。