We propose an explainable reinforcement learning (XRL) framework that analyzes an agent's history of interaction with the environment to extract interestingness elements that help explain its behavior. The framework relies on data readily available from standard RL algorithms, augmented with data that can easily be collected by the agent while learning. We describe how to create visual explanations of an agent's behavior in the form of short video-clips highlighting key interaction moments, based on the proposed elements. We also report on a user study where we evaluated the ability of humans in correctly perceiving the aptitude of agents with different characteristics, including their capabilities and limitations, given explanations automatically generated by our framework. The results show that the diversity of aspects captured by the different interestingness elements is crucial to help humans correctly identify the agents' aptitude in the task, and determine when they might need adjustments to improve their performance.

本文提出一个可解释的强化学习框架，通过分析代理与环境的交互历史来提取有助于解释其行为的有趣元素，从而创造可视化的摘要，评估用户的感知能力，结果表明，不同有趣元素捕捉到的方面的多样性对于帮助人类正确理解代理在执行任务中的优势和限制是至关重要的。

可解释性强化学习的趣味性元素：理解智能体的能力和限制