City-scale 3D point cloud is a promising way to express detailed and
complicated outdoor structures. It encompasses both the appearance and geometry
features of segmented city components, including cars, streets, and buildings,
that can be utilized for attractive applications such as user-interactive
navigation of autonomous vehicles and drones. However, compared to the
extensive text annotations available for images and indoor scenes, the scarcity
of text annotations for outdoor scenes poses a significant challenge for
achieving these applications. To tackle this problem, we introduce the
CityRefer dataset for city-level visual grounding. The dataset consists of 35k
natural language descriptions of 3D objects appearing in SensatUrban city
scenes and 5k landmarks labels synchronizing with OpenStreetMap. To ensure the
quality and accuracy of the dataset, all descriptions and labels in the
CityRefer dataset are manually verified. We also have developed a baseline
system that can learn encoded language descriptions, 3D object instances, and
geographical information about the city's landmarks to perform visual grounding
on the CityRefer dataset. To the best of our knowledge, the CityRefer dataset
is the largest city-level visual grounding dataset for localizing specific 3D
objects.

介绍了 CityRefer 数据集，该数据集是最大的城市级地理定位数据集，用于表示具体的 3D 物体，并开发了基线系统来执行视觉定位任务。

CityRefer：城市尺度点云数据的地理感知 3D 视觉引导数据集

CityRefer: Geography-aware 3D Visual Grounding Dataset on City-scale  Point Cloud Data

Navigating through unstructured environments is a basic capability of
intelligent creatures, and thus is of fundamental interest in the study and
development of artificial intelligence. Long-range navigation is a complex
cognitive task that relies on developing an internal representation of space,
grounded by recognisable landmarks and robust visual processing, that can
simultaneously support continuous self-localisation ("I am here") and a
representation of the goal ("I am going there"). Building upon recent research
that applies deep reinforcement learning to maze navigation problems, we
present an end-to-end deep reinforcement learning approach that can be applied
on a city scale. Recognising that successful navigation relies on integration
of general policies with locale-specific knowledge, we propose a dual pathway
architecture that allows locale-specific features to be encapsulated, while
still enabling transfer to multiple cities. We present an interactive
navigation environment that uses Google StreetView for its photographic content
and worldwide coverage, and demonstrate that our learning method allows agents
to learn to navigate multiple cities and to traverse to target destinations
that may be kilometres away. The project webpage this http URL contains
a video summarising our research and showing the trained agent in diverse city
environments and on the transfer task, the form to request the StreetLearn
dataset and links to further resources. The StreetLearn environment code is
available at this https URL

该论文提出一种基于深度强化学习的城市级别自主导航模型，采用 Google StreetView 进行环境交互和训练，具备可迁移性和区域特征一致性，实现了在多个城市环境下自主导航到目标位置。