Understanding the interactions of agents trained with deep reinforcement
learning is crucial for deploying agents in games or the real world. In the
former, unreasonable actions confuse players. In the latter, that effect is
even more significant, as unexpected behavior cause accidents with potentially
grave and long-lasting consequences for the involved individuals. In this work,
we propose using program synthesis to imitate reinforcement learning policies
after seeing a trajectory of the action sequence. Programs have the advantage
that they are inherently interpretable and verifiable for correctness. We adapt
the state-of-the-art program synthesis system DreamCoder for learning concepts
in grid-based environments, specifically, a navigation task and two miniature
versions of Atari games, Space Invaders and Asterix. By inspecting the
generated libraries, we can make inferences about the concepts the black-box
agent has learned and better understand the agent's behavior. We achieve the
same by visualizing the agent's decision-making process for the imitated
sequences. We evaluate our approach with different types of program
synthesizers based on a search-only method, a neural-guided search, and a
language model fine-tuned on code.

使用程序合成方法对深度强化学习代理进行模仿，以了解其学习的概念和决策过程。