In this article, we explore the potential of zero-shot Large Multimodal
Models (LMMs) in the domain of drone perception. We focus on person detection
and action recognition tasks and evaluate two prominent LMMs, namely YOLO-World
and GPT-4V(ision) using a publicly available dataset captured from aerial
views. Traditional deep learning approaches rely heavily on large and
high-quality training datasets. However, in certain robotic settings, acquiring
such datasets can be resource-intensive or impractical within a reasonable
timeframe. The flexibility of prompt-based Large Multimodal Models (LMMs) and
their exceptional generalization capabilities have the potential to
revolutionize robotics applications in these scenarios. Our findings suggest
that YOLO-World demonstrates good detection performance. GPT-4V struggles with
accurately classifying action classes but delivers promising results in
filtering out unwanted region proposals and in providing a general description
of the scenery. This research represents an initial step in leveraging LMMs for
drone perception and establishes a foundation for future investigations in this
area.

本文探讨了零样本大型多模态模型（Large Multimodal Models，LMMs）在无人机感知领域的潜力以及其在人体检测和动作识别任务中的应用。研究结果表明，YOLO-World 在检测性能上表现良好，而 GPT-4V 在准确分类动作类别上有困难，但在过滤不需要的区域提案和提供场景的总体描述方面取得了有希望的结果。该研究为利用 LMMs 进行无人机感知奠定了基础，并为进一步探索该领域提供了初始步骤。