Geospatial Copilots unlock unprecedented potential for performing Earth
Observation (EO) applications through natural language instructions. However,
existing agents rely on overly simplified single tasks and template-based
prompts, creating a disconnect with real-world scenarios. In this work, we
present GeoLLM-Engine, an environment for tool-augmented agents with intricate
tasks routinely executed by analysts on remote sensing platforms. We enrich our
environment with geospatial API tools, dynamic maps/UIs, and external
multimodal knowledge bases to properly gauge an agent's proficiency in
interpreting realistic high-level natural language commands and its functional
correctness in task completions. By alleviating overheads typically associated
with human-in-the-loop benchmark curation, we harness our massively parallel
engine across 100 GPT-4-Turbo nodes, scaling to over half a million diverse
multi-tool tasks and across 1.1 million satellite images. By moving beyond
traditional single-task image-caption paradigms, we investigate
state-of-the-art agents and prompting techniques against long-horizon prompts.

地理空间联合指挥系统通过自然语言指令释放了执行地球观测应用的前所未有的潜力。本研究介绍了 GeoLLM-Engine，这是一个环境，用于工具增强的代理人执行遥感平台上由分析师经常执行的复杂任务。我们丰富了我们的环境，配备了地理空间 API 工具、动态地图 / UI 和外部多模态知识库，以准确评估代理人在解释真实高级自然语言命令和任务完成的功能正确性方面的能力。通过减轻与人在环回式基准策划相关的开销，我们利用 100 个 GPT-4-Turbo 节点的大规模并行计算引擎，覆盖了超过 50 万个多样化的多工具任务和 110 万幅卫星图像。通过超越传统的单任务图像描述范式，我们研究了最先进的代理人和提示技术在面对长期视角提示时的性能。