Unmanned Combat Aerial Vehicle (UCAV) dogfight, which refers to a fight
between two or more UCAVs usually at close quarters, plays a decisive role on
the aerial battlefields. With the evolution of artificial intelligence,
dogfight progressively transits towards intelligent and autonomous modes.
However, the development of autonomous dogfight policy learning is hindered by
challenges such as weak exploration capabilities, low learning efficiency, and
unrealistic simulated environments. To overcome these challenges, this paper
proposes a novel imitative reinforcement learning framework, which efficiently
leverages expert data while enabling autonomous exploration. The proposed
framework not only enhances learning efficiency through expert imitation, but
also ensures adaptability to dynamic environments via autonomous exploration
with reinforcement learning. Therefore, the proposed framework can learn a
successful dogfight policy of 'pursuit-lock-launch' for UCAVs. To support
data-driven learning, we establish a dogfight environment based on the
Harfang3D sandbox, where we conduct extensive experiments. The results indicate
that the proposed framework excels in multistage dogfight, significantly
outperforms state-of-the-art reinforcement learning and imitation learning
methods. Thanks to the ability of imitating experts and autonomous exploration,
our framework can quickly learn the critical knowledge in complex aerial combat
tasks, achieving up to a 100% success rate and demonstrating excellent
robustness.

本文提出了一种新颖的模仿式强化学习框架，通过高效利用专家数据进行自主探索，不仅提高了学习效率，还通过强化学习实现了对动态环境的适应性，在无人战斗机的领域中学习到了成功的战斗策略。在基于 Harfang3D 沙盒环境上的实验中，我们的框架在多阶段空战中表现出色，显著优于现有的强化学习和模仿学习方法，具备模仿专家和自主探索的能力，能够快速学习复杂的空中作战任务中的关键知识，实现高达 100％的成功率并展示出极佳的稳健性。