This technical report presents the 1st winning model for UG2+, a task in CVPR
2024 UAV Tracking and Pose-Estimation Challenge. This challenge faces
difficulties in drone detection,
UAV-type classification and 2D/3D trajectory estimation in extreme weather
conditions with multi-modal sensor information, including stereo vision,
various Lidars, Radars, and audio arrays. Leveraging this information, we
propose a multi-modal UAV detection, classification, and 3D tracking method for
accurate UAV classification and tracking. A novel classification pipeline which
incorporates sequence fusion, region of interest (ROI) cropping, and keyframe
selection is proposed. Our system integrates cutting-edge classification
techniques and sophisticated post-processing steps to boost accuracy and
robustness. The designed pose estimation pipeline incorporates three modules:
dynamic points analysis, a multi-object tracker, and trajectory completion
techniques. Extensive experiments have validated the effectiveness and
precision of our approach. In addition, we also propose a novel dataset
pre-processing method and conduct a comprehensive ablation study for our
design. We finally achieved the best performance in the classification and
tracking of the MMUAD dataset. The code and configuration of our method are
available at this https URL

该研究提出了一种多模态无人机追踪和位姿估计方法，通过利用多模态传感器信息，包括立体视觉、各种雷达和音频阵列，以及针对无人机检测、分类和 3D 跟踪的困难，提高了准确度和稳健性，并在 MMUAD 数据集的分类和跟踪任务中取得了最好的性能。

多模态无人机检测、分类和跟踪算法 -- CVPR 2024 UG2 挑战技术报告

Multi-Modal UAV Detection, Classification and Tracking Algorithm --  Technical Report for CVPR 2024 UG2 Challenge

This paper presents a bonobo detection and classification pipeline built from
the commonly used machine learning methods. Such application is motivated by
the need to test bonobos in their enclosure using touch screen devices without
human assistance. This work introduces a newly acquired dataset based on bonobo
recordings generated semi-automatically. The recordings are weakly labelled and
fed to a macaque detector in order to spatially detect the individual present
in the video. Handcrafted features coupled with different classification
algorithms and deep-learning methods using a ResNet architecture are
investigated for bonobo identification. Performance is compared in terms of
classification accuracy on the splits of the database using different data
separation methods. We demonstrate the importance of data preparation and how a
wrong data separation can lead to false good results. Finally, after a
meaningful separation of the data, the best classification performance is
obtained using a fine-tuned ResNet model and reaches 75% of accuracy.

通过使用常用的机器学习方法，该研究论文介绍了一个基于棕猿检测和分类的流程，旨在通过触摸屏设备在棕猿围栏内无需人工辅助地进行测试。使用半自动方式生成的棕猿录音数据集被引入。这些录音数据具有弱标签，并被输入到猕猴检测器中以空间上检测视频中的个体。研究了手工特征结合不同的分类算法和使用 ResNet 架构的深度学习方法来进行棕猿识别。性能通过使用不同的数据分离方法对数据库进行分割后的分类准确性进行比较。我们展示了数据准备的重要性以及错误的数据分离如何导致虚假的良好结果。最后，在对数据进行有意义的分离之后，使用经过微调的 ResNet 模型获得了最佳的分类性能，准确率达到了 75%。