Remote sensing semantic segmentation (RSS) is an essential task in Earth
Observation missions. Due to data privacy concerns, high-quality remote sensing
images with annotations cannot be well shared among institutions, making it
difficult to fully utilize RSS data to train a generalized model. Federated
Learning (FL), a privacy-preserving collaborative learning technology, is a
potential solution. However, the current research on how to effectively apply
FL in RSS is still scarce and requires further investigation. Remote sensing
images in various institutions often exhibit strong geographical heterogeneity.
More specifically, it is reflected in terms of class-distribution heterogeneity
and object-appearance heterogeneity. Unfortunately, most existing FL studies
show inadequate focus on geographical heterogeneity, thus leading to
performance degradation in the global model. Considering the aforementioned
issues, we propose a novel Geographic Heterogeneity-Aware Federated Learning
(GeoFed) framework to address privacy-preserving RSS. Through Global Feature
Extension and Tail Regeneration modules, class-distribution heterogeneity is
alleviated. Additionally, we design an Essential Feature Mining strategy to
alleviate object-appearance heterogeneity by constructing essential features.
Extensive experiments on three datasets (i.e., FBP, CASID, Inria) show that our
GeoFed consistently outperforms the current state-of-the-art methods. The code
will be available publicly.

通过 GeoFed 框架，我们提出了一种解决隐私保护遥感语义分割任务的方法，通过全局特征扩展和尾部重构模块来减轻分类分布的异质性，同时设计了一个关键特征挖掘策略来减轻目标外观的异质性。在三个数据集上进行的大量实验证明，我们的 GeoFed 方法始终优于目前的最优方法。

数据岛之桥：针对协作遥感语义分割的地理异质感知联邦学习

Bridging Data Islands: Geographic Heterogeneity-Aware Federated Learning  for Collaborative Remote Sensing Semantic Segmentation

The advent of high-resolution multispectral/hyperspectral sensors, LiDAR DSM
(Digital Surface Model) information and many others has provided us with an
unprecedented wealth of data for Earth Observation. Multimodal AI seeks to
exploit those complementary data sources, particularly for complex tasks like
semantic segmentation. While specialized architectures have been developed,
they are highly complicated via significant effort in model design, and require
considerable re-engineering whenever a new modality emerges. Recent trends in
general-purpose multimodal networks have shown great potential to achieve
state-of-the-art performance across multiple multimodal tasks with one unified
architecture. In this work, we investigate the performance of PerceiverIO, one
in the general-purpose multimodal family, in the remote sensing semantic
segmentation domain. Our experiments reveal that this ostensibly universal
network struggles with object scale variation in remote sensing images and
fails to detect the presence of cars from a top-down view. To address these
issues, even with extreme class imbalance issues, we propose a spatial and
volumetric learning component. Specifically, we design a UNet-inspired module
that employs 3D convolution to encode vital local information and learn
cross-modal features simultaneously, while reducing network computational
burden via the cross-attention mechanism of PerceiverIO. The effectiveness of
the proposed component is validated through extensive experiments comparing it
with other methods such as 2D convolution, and dual local module (\ie the
combination of Conv2D 1x1 and Conv2D 3x3 inspired by UNetFormer). The proposed
method achieves competitive results with specialized architectures like
UNetFormer and SwinUNet, showing its potential to minimize network architecture
engineering with a minimal compromise on the performance.

在遥感语义分割领域中，本研究调查了 PerceiverIO 等普适性多模态网络在处理遥感图像中的物体尺度变化和车辆检测方面的表现，并提出一种空间和体积学习组件来解决这些问题。通过广泛的实验证明，该方法在与 UNetFormer 和 SwinUNet 等专用架构相比具有竞争力的结果，显示出最小化网络架构工程且性能不受明显影响的潜力。