We approach instantaneous mapping, converting images to a top-down view of
the world, as a translation problem. We show how a novel form of transformer
network can be used to map from images and video directly to an overhead map or
bird's-eye-view (BEV) of the world, in a single end-to-end network. We assume a
1-1 correspondence between a vertical scanline in the image, and rays passing
through the camera location in an overhead map. This lets us formulate map
generation from an image as a set of sequence-to-sequence translations. Posing
the problem as translation allows the network to use the context of the image
when interpreting the role of each pixel. This constrained formulation, based
upon a strong physical grounding of the problem, leads to a restricted
transformer network that is convolutional in the horizontal direction only. The
structure allows us to make efficient use of data when training, and obtains
state-of-the-art results for instantaneous mapping of three large-scale
datasets, including a 15% and 30% relative gain against existing best
performing methods on the nuScenes and Argoverse datasets, respectively. We
make our code available on
this https URL

通过使用新形式的变压器网络，直接将图像或视频转换为俯视图或鸟瞰视图，从而获得即时映射，通过顺序到顺序的翻译来生成地图，利用图像的背景信息来解析每个像素的作用，该算法在三个大规模数据集上的表现达到了最佳水平。