In this paper, a novel Unified Multi-Task Learning Framework of Real-Time
Drone Supervision for Crowd Counting (MFCC) is proposed, which utilizes an
image fusion network architecture to fuse images from the visible and thermal
infrared image, and a crowd counting network architecture to estimate the
density map. The purpose of our framework is to fuse two modalities, including
visible and thermal infrared images captured by drones in real-time, that
exploit the complementary information to accurately count the dense population
and then automatically guide the flight of the drone to supervise the dense
crowd. To this end, we propose the unified multi-task learning framework for
crowd counting for the first time and re-design the unified training loss
functions to align the image fusion network and crowd counting network. We also
design the Assisted Learning Module (ALM) to fuse the density map feature to
the image fusion encoder process for learning the counting features. To improve
the accuracy, we propose the Extensive Context Extraction Module (ECEM) that is
based on a dense connection architecture to encode multi-receptive-fields
contextual information and apply the Multi-domain Attention Block (MAB) for
concerning the head region in the drone view. Finally, we apply the prediction
map to automatically guide the drones to supervise the dense crowd. The
experimental results on the DroneRGBT dataset show that, compared with the
existing methods, ours has comparable results on objective evaluations and an
easier training process.

本研究提出了一种新的联合多任务学习框架，用于实时监控人群计数的无人机监控。该框架利用图像融合网络结构来融合可见光和热红外图像，并使用人群计数网络结构来估计密度图。结果表明，与现有方法相比，该方法在客观评估和训练过程中具有可比较的结果。