Multi-modal large language models (MLLMs) have demonstrated remarkable
success in vision and visual-language tasks within the natural image domain.
Owing to the significant diversities between the natural image and RS image
hinder the development of MLLMs in the remote sensing (RS) domain. Currently,
the unified and powerful MLLM capable of various RS visual tasks is still
under-explored. To fill the gap, a pioneer MLLM called EarthGPT is proposed for
universal RS image comprehension, which integrates various multi-sensor RS
interpretation tasks uniformly. More importantly, a large-scale multi-sensor
multi-modal RS instruction-following dataset named MMRS is carefully
constructed, which comprises 1005.842k image-text pairs based on 34 existing
diverse RS datasets and includes multi-sensor images such as optical, synthetic
aperture radar (SAR), and infrared. The MMRS addresses the issue of MLLMs
lacking RS expert knowledge and stimulates the development of MMLMs in the RS
domain. Extensive experiments demonstrate the EarthGPT's superior performance
in various RS visual interpretation tasks compared with the other specialist
models and MLLMs, which proves the effectiveness of the proposed EarthGPT and
provides a versatile paradigm for open-set reasoning tasks.

提出了一个先驱性的多模态大型语言模型（MLLM）EarthGPT，该模型集成了各种多传感器遥感图像解释任务，通过构建大规模的多传感器多模态遥感指令追踪数据集 MMRS，解决了 MLLM 在遥感领域缺乏专业知识的问题，并在各种遥感图像解释任务上展现出其卓越的性能，并为开放式推理任务提供了一种多功能的范例。