In the field of robotics and computer vision, efficient and accurate semantic
mapping remains a significant challenge due to the growing demand for
intelligent machines that can comprehend and interact with complex
environments. Conventional panoptic mapping methods, however, are limited by
predefined semantic classes, thus making them ineffective for handling novel or
unforeseen objects. In response to this limitation, we introduce the Unified
Promptable Panoptic Mapping (UPPM) method. UPPM utilizes recent advances in
foundation models to enable real-time, on-demand label generation using natural
language prompts. By incorporating a dynamic labeling strategy into traditional
panoptic mapping techniques, UPPM provides significant improvements in
adaptability and versatility while maintaining high performance levels in map
reconstruction. We demonstrate our approach on real-world and simulated
datasets. Results show that UPPM can accurately reconstruct scenes and segment
objects while generating rich semantic labels through natural language
interactions. A series of ablation experiments validated the advantages of
foundation model-based labeling over fixed label sets.

通过使用自然语言提示词，我们介绍了统一的可提示全景映射（UPPM）方法，将动态标注策略与传统的全景映射技术相结合，实现实时、按需的标签生成，并在场景重建中表现出良好的适应性和多功能性。