Reinforcement learning (RL) algorithms often struggle to learn policies that
generalize to novel situations due to issues such as causal confusion,
overfitting to irrelevant factors, and failure to isolate control of state
factors. These issues stem from a common source: a failure to accurately
identify and exploit state-specific causal relationships in the environment.
While some prior works in RL aim to identify these relationships explicitly,
they rely on informal domain-specific heuristics such as spatial and temporal
proximity. Actual causality offers a principled and general framework for
determining the causes of particular events. However, existing definitions of
actual cause often attribute causality to a large number of events, even if
many of them rarely influence the outcome. Prior work on actual causality
proposes normality as a solution to this problem, but its existing
implementations are challenging to scale to complex and continuous-valued RL
environments. This paper introduces functional actual cause (FAC), a framework
that uses context-specific independencies in the environment to restrict the
set of actual causes. We additionally introduce Joint Optimization for Actual
Cause Inference (JACI), an algorithm that learns from observational data to
infer functional actual causes. We demonstrate empirically that FAC agrees with
known results on a suite of examples from the actual causality literature, and
JACI identifies actual causes with significantly higher accuracy than existing
heuristic methods in a set of complex, continuous-valued environments.

通过使用功能实际因果 (FAC) 和联合优化实际因果推断 (JACI) 算法，本研究在复杂连续数值环境中展示了 FAC 与实际因果文献中已知结果的一致性，并且 JACI 在识别实际原因的准确性上明显优于现有的启发式方法。