Multi-robot path finding in dynamic environments is a highly challenging
classic problem. In the movement process, robots need to avoid collisions with
other moving robots while minimizing their travel distance. Previous methods
for this problem either continuously replan paths using heuristic search
methods to avoid conflicts or choose appropriate collision avoidance strategies
based on learning approaches. The former may result in long travel distances
due to frequent replanning, while the latter may have low learning efficiency
due to low sample exploration and utilization, and causing high training costs
for the model. To address these issues, we propose a path planning method,
MAPPOHR, which combines heuristic search, empirical rules, and multi-agent
reinforcement learning. The method consists of two layers: a real-time planner
based on the multi-agent reinforcement learning algorithm, MAPPO, which embeds
empirical rules in the action output layer and reward functions, and a
heuristic search planner used to create a global guiding path. During movement,
the heuristic search planner replans new paths based on the instructions of the
real-time planner. We tested our method in 10 different conflict scenarios. The
experiments show that the planning performance of MAPPOHR is better than that
of existing learning and heuristic methods. Due to the utilization of empirical
knowledge and heuristic search, the learning efficiency of MAPPOHR is higher
than that of existing learning methods.

提出了一种结合启发式搜索、经验法则和多智能体强化学习的路径规划方法 MAPPOHR，将启发式搜索用于生成全局路径，用经验法则和奖励函数激励实时规划器，用多智能体强化学习算法实现实时规划，实验证明该方法比传统学习和启发式方法的规划性能更好且学习效率更高。