Semantic grids can be useful representations of the scene around an
autonomous system. By having information about the layout of the space around
itself, a robot can leverage this type of representation for crucial tasks such
as navigation or tracking. By fusing information from multiple sensors,
robustness can be increased and the computational load for the task can be
lowered, achieving real time performance. Our multi-scale LiDAR-Aided
Perspective Transform network uses information available in point clouds to
guide the projection of image features to a top-view representation, resulting
in a relative improvement in the state of the art for semantic grid generation
for human (+8.67%) and movable object (+49.07%) classes in the nuScenes
dataset, as well as achieving results close to the state of the art for the
vehicle, drivable area and walkway classes, while performing inference at 25
FPS.

本研究提出了一种多尺度 LiDAR 辅助透视变换神经网络，利用点云信息将图像特征投影到俯视图中，从而在大幅度提升语义网格生成方面取得了实质性的进展，实现了 25FPS 的实时性能。

LAPTNet-FPN: 多尺度 LiDAR 辅助的投影变换网络 - 用于实时语义栅格预测

LAPTNet-FPN: Multi-scale LiDAR-aided Projective Transform Network for Real Time Semantic Grid Prediction

For autonomous robots to navigate a complex environment, it is crucial to
understand the surrounding scene both geometrically and semantically. Modern
autonomous robots employ multiple sets of sensors, including lidars, radars,
and cameras. Managing the different reference frames and characteristics of the
sensors, and merging their observations into a single representation
complicates perception. Choosing a single unified representation for all
sensors simplifies the task of perception and fusion. In this work, we present
an end-to-end pipeline that performs semantic segmentation and short term
prediction using a top-down representation. Our approach consists of an
ensemble of neural networks which take in sensor data from different sensor
modalities and transform them into a single common top-down semantic grid
representation. We find this representation favorable as it is agnostic to
sensor-specific reference frames and captures both the semantic and geometric
information for the surrounding scene. Because the modalities share a single
output representation, they can be easily aggregated to produce a fused output.
In this work we predict short-term semantic grids but the framework can be
extended to other tasks. This approach offers a simple, extensible, end-to-end
approach for multi-modal perception and prediction.

本文介绍了一种利用多模态传感器数据进行多模态感知和预测的方法，采用一种自上而下的语义网格表示形式，对不同模态的传感器数据进行统一的表达，使得预测和融合更加简单可行。