Drones equipped with cameras can significantly enhance human ability to
perceive the world because of their remarkable maneuverability in 3D space.
Ironically, object detection for drones has always been conducted in the 2D
image space, which fundamentally limits their ability to understand 3D scenes.
Furthermore, existing 3D object detection methods developed for autonomous
driving cannot be directly applied to drones due to the lack of deformation
modeling, which is essential for the distant aerial perspective with sensitive
distortion and small objects. To fill the gap, this work proposes a dual-view
detection system named DVDET to achieve aerial monocular object detection in
both the 2D image space and the 3D physical space. To address the severe view
deformation issue, we propose a novel trainable geo-deformable transformation
module that can properly warp information from the drone's perspective to the
BEV. Compared to the monocular methods for cars, our transformation includes a
learnable deformable network for explicitly revising the severe deviation. To
address the dataset challenge, we propose a new large-scale simulation dataset
named AM3D-Sim, generated by the co-simulation of AirSIM and CARLA, and a new
real-world aerial dataset named AM3D-Real, collected by DJI Matrice 300 RTK, in
both datasets, high-quality annotations for 3D object detection are provided.
Extensive experiments show that i) aerial monocular 3D object detection is
feasible; ii) the model pre-trained on the simulation dataset benefits
real-world performance, and iii) DVDET also benefits monocular 3D object
detection for cars. To encourage more researchers to investigate this area, we
will release the dataset and related code in
this https URL.

通过双视图检测系统 DVDET，提出了一种基于地理变形的训练模块，实现了 2D 图像空间和 3D 物理空间的空中单目物体检测，并利用新的数据集 AM3D，证明了模型的可行性及性能。