The intelligent interpretation of buildings plays a significant role in urban
planning and management, macroeconomic analysis, population dynamics, etc.
Remote sensing image building interpretation primarily encompasses building
extraction and change detection. However, current methodologies often treat
these two tasks as separate entities, thereby failing to leverage shared
knowledge. Moreover, the complexity and diversity of remote sensing image
scenes pose additional challenges, as most algorithms are designed to model
individual small datasets, thus lacking cross-scene generalization. In this
paper, we propose a comprehensive remote sensing image building understanding
model, termed RSBuilding, developed from the perspective of the foundation
model. RSBuilding is designed to enhance cross-scene generalization and task
universality. Specifically, we extract image features based on the prior
knowledge of the foundation model and devise a multi-level feature sampler to
augment scale information. To unify task representation and integrate image
spatiotemporal clues, we introduce a cross-attention decoder with task prompts.
Addressing the current shortage of datasets that incorporate annotations for
both tasks, we have developed a federated training strategy to facilitate
smooth model convergence even when supervision for some tasks is missing,
thereby bolstering the complementarity of different tasks. Our model was
trained on a dataset comprising up to 245,000 images and validated on multiple
building extraction and change detection datasets. The experimental results
substantiate that RSBuilding can concurrently handle two structurally distinct
tasks and exhibits robust zero-shot generalization capabilities.

提出了一种综合的遥感图像建筑理解模型，具有跨场景泛化和任务普适性，可以同时处理建筑提取和变化检测等两个结构不同的任务，并展现出鲁棒的零样本推广能力。