Understanding the agent's learning process, particularly the factors that
contribute to its success or failure post-training, is crucial for
comprehending the rationale behind the agent's decision-making process. Prior
methods clarify the learning process by creating a structural causal model
(SCM) or visually representing the distribution of value functions.
Nevertheless, these approaches have constraints as they exclusively function in
2D-environments or with uncomplicated transition dynamics. Understanding the
agent's learning process in complicated environments or tasks is more
challenging. In this paper, we propose REVEAL-IT, a novel framework for
explaining the learning process of an agent in complex environments. Initially,
we visualize the policy structure and the agent's learning process for various
training tasks. By visualizing these findings, we can understand how much a
particular training task or stage affects the agent's performance in test.
Then, a GNN-based explainer learns to highlight the most important section of
the policy, providing a more clear and robust explanation of the agent's
learning process. The experiments demonstrate that explanations derived from
this framework can effectively help in the optimization of the

在本文中，我们提出了 REVEAL-IT 框架，用于解释复杂环境中代理人的学习过程。我们通过可视化策略结构和代理人在各种训练任务中的学习过程来理解一个特定的训练任务或阶段对代理人在测试中的性能有多大影响。然后，基于图神经网络的解释器学习突出策略中最重要的部分，提供更清晰和更强大的解释代理人学习过程的工具。实验证明，从该框架获得的解释能够有效帮助优化。