Deep policy networks enable robots to learn behaviors to solve various
real-world complex tasks in an end-to-end fashion. However, they lack
transparency to provide the reasons of actions. Thus, such a black-box model
often results in low reliability and disruptive actions during the deployment
of the robot in practice. To enhance its transparency, it is important to
explain robot behaviors by considering the extent to which each input feature
contributes to determining a given action. In this paper, we present an
explicit analysis of deep policy models through input attribution methods to
explain how and to what extent each input feature affects the decisions of the
robot policy models. To this end, we present two methods for applying input
attribution methods to robot policy networks: (1) we measure the importance
factor of each joint torque to reflect the influence of the motor torque on the
end-effector movement, and (2) we modify a relevance propagation method to
handle negative inputs and outputs in deep policy networks properly. To the
best of our knowledge, this is the first report to identify the dynamic changes
of input attributions of multi-modal sensor inputs in deep policy networks
online for robotic manipulation.

通过应用输入归因方法对机器人策略网络进行可解释性分析，首次在线动态改变多模式传感器输入在深度策略网络中的输入归因，解释了机器人行为的影响因素。