Earth vision research typically focuses on extracting geospatial object
locations and categories but neglects the exploration of relations between
objects and comprehensive reasoning. Based on city planning needs, we develop a
multi-modal multi-task VQA dataset (EarthVQA) to advance relational
reasoning-based judging, counting, and comprehensive analysis. The EarthVQA
dataset contains 6000 images, corresponding semantic masks, and 208,593 QA
pairs with urban and rural governance requirements embedded. As objects are the
basis for complex relational reasoning, we propose a Semantic OBject Awareness
framework (SOBA) to advance VQA in an object-centric way. To preserve refined
spatial locations and semantics, SOBA leverages a segmentation network for
object semantics generation. The object-guided attention aggregates object
interior features via pseudo masks, and bidirectional cross-attention further
models object external relations hierarchically. To optimize object counting,
we propose a numerical difference loss that dynamically adds difference
penalties, unifying the classification and regression tasks. Experimental
results show that SOBA outperforms both advanced general and remote sensing
methods. We believe this dataset and framework provide a strong benchmark for
Earth vision's complex analysis. The project page is at
this https URL

基于城市规划需求，我们开发了一个多模态多任务的 VQA 数据集（EarthVQA），旨在推动基于关系推理的判断、计数和综合分析。SOBA 框架以物体为中心，提出了一个语义物体感知的方法，通过对象语义生成和对象引导的注意力，以及双向交叉注意力，对复杂关系推理进行了先进的视觉问答模型优化。实验证明，SOBA 优于现有的通用和遥感方法，该数据集和框架为地球视觉的复杂分析提供了一个强大的基准。