Multi-modal large language models (MLLMs) have demonstrated remarkable success in vision and visual-language tasks within the natural image domain. Owing to the significant diversities between the natural image and RS image hinder the development of MLLMs in the remote sensing (RS) domain. Currently, the unified and powerful MLLM capable of various RS visual tasks is still under-explored. To fill the gap, a pioneer MLLM called EarthGPT is proposed for universal RS image comprehension, which integrates various multi-sensor RS interpretation tasks uniformly. More importantly, a large-scale multi-sensor multi-modal RS instruction-following dataset named MMRS is carefully constructed, which comprises 1005.842k image-text pairs based on 34 existing diverse RS datasets and includes multi-sensor images such as optical, synthetic aperture radar (SAR), and infrared. The MMRS addresses the issue of MLLMs lacking RS expert knowledge and stimulates the development of MMLMs in the RS domain. Extensive experiments demonstrate the EarthGPT's superior performance in various RS visual interpretation tasks compared with the other specialist models and MLLMs, which proves the effectiveness of the proposed EarthGPT and provides a versatile paradigm for open-set reasoning tasks.

提出了一个先驱性的多模态大型语言模型（MLLM）EarthGPT，该模型集成了各种多传感器遥感图像解释任务，通过构建大规模的多传感器多模态遥感指令追踪数据集MMRS，解决了MLLM在遥感领域缺乏专业知识的问题，并在各种遥感图像解释任务上展现出其卓越的性能，并为开放式推理任务提供了一种多功能的范例。

EarthGPT：遥感领域多感知器图像理解的通用多模态大型语言模型