Reinforcement learning (RL) is applied in a wide variety of fields. RL
enables agents to learn tasks autonomously by interacting with the environment.
The more critical the tasks are, the higher the demand for the robustness of
the RL systems. Causal RL combines RL and causal inference to make RL more
robust. Causal RL agents use a causal representation to capture the invariant
causal mechanisms that can be transferred from one task to another. Currently,
there is limited research in Causal RL, and existing solutions are usually not
complete or feasible for real-world applications. In this work, we propose
CausalCF, the first complete Causal RL solution incorporating ideas from Causal
Curiosity and CoPhy. Causal Curiosity provides an approach for using
interventions, and CoPhy is modified to enable the RL agent to perform
counterfactuals. We apply CausalCF to complex robotic tasks and show that it
improves the RL agent's robustness using a realistic simulation environment
called CausalWorld.

本研究提出 CausalCF，将因果推理与强化学习相结合，使得该模型在复杂任务上更加稳健，同时，实验表明 CausalCF 已经成为了第一个完整的能够将 Causal Curiosity 和 CoPhy 思想融入的因果强化学习解决方案。