Semantic 2D maps are commonly used by humans and machines for navigation
purposes, whether it's walking or driving. However, these maps have
limitations: they lack detail, often contain inaccuracies, and are difficult to
create and maintain, especially in an automated fashion. Can we use raw imagery
to automatically create better maps that can be easily interpreted by both
humans and machines? We introduce SNAP, a deep network that learns rich neural
2D maps from ground-level and overhead images. We train our model to align
neural maps estimated from different inputs, supervised only with camera poses
over tens of millions of StreetView images. SNAP can resolve the location of
challenging image queries beyond the reach of traditional methods,
outperforming the state of the art in localization by a large margin. Moreover,
our neural maps encode not only geometry and appearance but also high-level
semantics, discovered without explicit supervision. This enables effective
pre-training for data-efficient semantic scene understanding, with the
potential to unlock cost-efficient creation of more detailed maps.

通过使用人工智能技术，本文介绍了一种名为 SNAP 的新型神经网络，可以自动创建丰富的 2D 地图，并具有更高级的语义信息。该方法可以解决创建和维护 2D 地图的困难，并在定位方面取得了业界领先的表现。

SNAP：自监督神经地图用于视觉定位与语义理解

SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic  Understanding

Current methods for training recurrent neural networks are based on
backpropagation through time, which requires storing a complete history of
network states, and prohibits updating the weights `online' (after every
timestep). Real Time Recurrent Learning (RTRL) eliminates the need for history
storage and allows for online weight updates, but does so at the expense of
computational costs that are quartic in the state size. This renders RTRL
training intractable for all but the smallest networks, even ones that are made
highly sparse.
We introduce the Sparse n-step Approximation (SnAp) to the RTRL influence
matrix, which only keeps entries that are nonzero within n steps of the
recurrent core. SnAp with n=1 is no more expensive than backpropagation, and we
find that it substantially outperforms other RTRL approximations with
comparable costs such as Unbiased Online Recurrent Optimization. For highly
sparse networks, SnAp with n=2 remains tractable and can outperform
backpropagation through time in terms of learning speed when updates are done
online. SnAp becomes equivalent to RTRL when n is large.

介绍了 Sparse n-step Approximation (SnAp) 来优化 Real Time Recurrent Learning (RTRL) 的影响矩阵，在保持计算成本低的同时提高了网络的学习速度，特别是在稀疏网络时，n=2 时的 SnAp 能够优于 backpropagation in terms of learning speed when updates are done online。