The generic large Vision-Language Models (VLMs) is rapidly developing, but
still perform poorly in Remote Sensing (RS) domain, which is due to the unique
and specialized nature of RS imagery and the comparatively limited spatial
perception of current VLMs. Existing Remote Sensing specific Vision Language
Models (RSVLMs) still have considerable potential for improvement, primarily
owing to the lack of large-scale, high-quality RS vision-language datasets. We
constructed HqDC-1.4M, the large scale High quality and Detailed Captions for
RS images, containing 1.4 million image-caption pairs, which not only enhance
the RSVLM's understanding of RS images but also significantly improve the
model's spatial perception abilities, such as localization and counting,
thereby increasing the helpfulness of the RSVLM. Moreover, to address the
inevitable "hallucination" problem in RSVLM, we developed RSSA, the first
dataset aimed at enhancing the Self-Awareness capability of RSVLMs. By
incorporating a variety of unanswerable questions into typical RS visual
question-answering tasks, RSSA effectively improves the truthfulness and
reduces the hallucinations of the model's outputs, thereby enhancing the
honesty of the RSVLM. Based on these datasets, we proposed the H2RSVLM, the
Helpful and Honest Remote Sensing Vision Language Model. H2RSVLM has achieved
outstanding performance on multiple RS public datasets and is capable of
recognizing and refusing to answer the unanswerable questions, effectively
mitigating the incorrect generations. We will release the code, data and model
weights at this https URL .

该研究构建了包含 1.4 百万图像 - 字幕对的大规模高质量细节遥感图像数据集 HqDC-1.4M，提供了对遥感图像更好的理解并显著提高了模型的空间感知能力；同时，开发了 RSSA 数据集，通过将无法回答的问题纳入典型的遥感视觉问答任务中，有效改善了模型输出的真实性和幻觉现象，增强了遥感视觉语言模型的诚实度；基于这些数据集，提出了 H2RSVLM，该模型在多个公共遥感数据集上取得了出色的性能，并能识别和拒绝无法回答的问题，有效减轻了错误的生成。