In large-scale disaster events, the planning of optimal rescue routes depends
on the object detection ability at the disaster scene, with one of the main
challenges being the presence of dense and occluded objects. Existing methods,
which are typically based on the RGB modality, struggle to distinguish targets
with similar colors and textures in crowded environments and are unable to
identify obscured objects. To this end, we first construct two multimodal dense
and occlusion vehicle detection datasets for large-scale events, utilizing RGB
and height map modalities. Based on these datasets, we propose a multimodal
collaboration network for dense and occluded vehicle detection, MuDet for
short. MuDet hierarchically enhances the completeness of discriminable
information within and across modalities and differentiates between simple and
complex samples. MuDet includes three main modules: Unimodal Feature
Hierarchical Enhancement (Uni-Enh), Multimodal Cross Learning (Mul-Lea), and
Hard-easy Discriminative (He-Dis) Pattern. Uni-Enh and Mul-Lea enhance the
features within each modality and facilitate the cross-integration of features
from two heterogeneous modalities. He-Dis effectively separates densely
occluded vehicle targets with significant intra-class differences and minimal
inter-class differences by defining and thresholding confidence values, thereby
suppressing the complex background. Experimental results on two re-labeled
multimodal benchmark datasets, the 4K-SAI-LCS dataset, and the ISPRS Potsdam
dataset, demonstrate the robustness and generalization of the MuDet. The codes
of this work are available openly at https://github.com/Shank2358/MuDet.

在大规模灾难事件中，构建最优救援路径的规划依赖于在灾难现场的目标检测能力，其中主要挑战之一是存在密集和遮挡的目标。该研究提出了一种多模态协作网络（MuDet）用于密集和遮挡的车辆检测，通过构建多模态数据集并采用三个主要模块来增强不同模态的特征，并通过定义和阈值化置信度值来有效分离复杂背景中密集遮挡的车辆目标。实验结果表明 MuDet 的鲁棒性和通用性。