This paper introduces an innovative approach to road network generation
through the utilization of a multi-modal Large Language Model (LLM). Our model
is specifically designed to process aerial images of road layouts and produce
detailed, navigable road networks within the input images. The core innovation
of our system lies in the unique training methodology employed for the large
language model to generate road networks as its output. This approach draws
inspiration from the BLIP-2 architecture arXiv:2301.12597, leveraging
pre-trained frozen image encoders and large language models to create a
versatile multi-modal LLM.
Our work also offers an alternative to the reasoning segmentation method
proposed in the LISA paper arXiv:2308.00692. By training the large language
model with our approach, the necessity for generating binary segmentation
masks, as suggested in the LISA paper arXiv:2308.00692, is effectively
eliminated. Experimental results underscore the efficacy of our multi-modal LLM
in providing precise and valuable navigational guidance. This research
represents a significant stride in bolstering autonomous navigation systems,
especially in road network scenarios, where accurate guidance is of paramount
importance.

这篇论文介绍了一种创新的道路网络生成方法，通过使用多模式大型语言模型 (LLM) 处理航空图像的道路布局，并在输入图像中生成详细的可导航道路网络。我们的模型通过独特的训练方法创造了道路网络，采用了 BLIP-2 架构与预训练的冻结图像编码器和大型语言模型相结合的方法创建了一个多功能的多模式 LLM，实验证明了我们的模型在提供精确和有价值的导航指引方面的有效性，这是增强自主导航系统的重要进展。