This report details the method of the winning entry of the AVDN Challenge in
ICCV 2023. The competition addresses the Aerial Navigation from Dialog History
(ANDH) task, which requires a drone agent to associate dialog history with
aerial observations to reach the destination. For better cross-modal grounding
abilities of the drone agent, we propose a Target-Grounded Graph-Aware
Transformer (TG-GAT) framework. Concretely, TG-GAT first leverages a
graph-aware transformer to capture spatiotemporal dependency, which is
beneficial for navigation state tracking and robust action planning. TG-GAT
first leverages a graph-aware transformer to capture spatiotemporal
dependencies for more robust action planning. In addition, an auxiliary visual
grounding task is devised to boost the agent's awareness of referred landmarks.
Moreover, a hybrid augmentation strategy based on large language models is
utilized to mitigate data scarcity limitations. Our TG-GAT framework won the
AVDN Challenge 2023, with 2.2% and 3.0% absolute improvements over the baseline
on SPL and SR metrics, respectively. The code is available at
this https URL

本研究提出了一种目标感知的图注意力变换器（TG-GAT）框架，用于无人机代理相关对话历史与航空观测的关联，以实现导航任务，并通过大型语言模型实现辅助视觉以及数据增强策略，有效提高导航性能。