Although deep reinforcement learning has been shown to be effective, the model's black-box nature presents barriers to direct policy interpretation. To address this problem, we propose a neuro-symbolic approach called neural DNF-MT for end-to-end policy learning. The differentiable nature of the neural DNF-MT model enables the use of deep actor-critic algorithms for training. At the same time, its architecture is designed so that trained models can be directly translated into interpretable policies expressed as standard (bivalent or probabilistic) logic programs. Moreover, additional layers can be included to extract abstract features from complex observations, acting as a form of predicate invention. The logic representations are highly interpretable, and we show how the bivalent representations of deterministic policies can be edited and incorporated back into a neural model, facilitating manual intervention and adaptation of learned policies. We evaluate our approach on a range of tasks requiring learning deterministic or stochastic behaviours from various forms of observations. Our empirical results show that our neural DNF-MT model performs at the level of competing black-box methods whilst providing interpretable policies.

本研究解决了深度强化学习中的策略可解释性问题，提出了一种名为神经DNF-MT的新型神经符号方法，能够实现端到端的策略学习。通过将模型训练与标准逻辑程序相结合，研究展示了如何将确定性政策的双值表示进行编辑，并将其融入神经模型中，从而实现对所学习策略的手动干预和适应。

神经DNF-MT：一种用于学习可解释和可编辑策略的神经符号方法