Many dynamic processes, including common scenarios in robotic control and
reinforcement learning (RL), involve a set of interacting subprocesses. Though
the subprocesses are not independent, their interactions are often sparse, and
the dynamics at any given time step can often be decomposed into locally
independent causal mechanisms. Such local causal structures can be leveraged to
improve the sample efficiency of sequence prediction and off-policy
reinforcement learning. We formalize this by introducing local causal models
(LCMs), which are induced from a global causal model by conditioning on a
subset of the state space. We propose an approach to inferring these structures
given an object-oriented state representation, as well as a novel algorithm for
Counterfactual Data Augmentation (CoDA). CoDA uses local structures and an
experience replay to generate counterfactual experiences that are causally
valid in the global model. We find that CoDA significantly improves the
performance of RL agents in locally factored tasks, including the
batch-constrained and goal-conditioned settings.

本文介绍了一种利用局部因果模型（local causal models，LCMs）、逆因数据增强（Counterfactual Data Augmentation，CoDA）等方法来提高强化学习（reinforcement learning，RL）算法采样效率的方法。实验表明，在其所擅长的分解式任务中，该方法能显著提高算法的表现。