The core task of any autonomous driving system is to transform sensory inputs
into driving commands. In end-to-end driving, this is achieved via a neural
network, with one or multiple cameras as the most commonly used input and
low-level driving command, e.g. steering angle, as output. However,
depth-sensing has been shown in simulation to make the end-to-end driving task
easier. On a real car, combining depth and visual information can be
challenging, due to the difficulty of obtaining good spatial and temporal
alignment of the sensors. To alleviate alignment problems, Ouster LiDARs can
output surround-view LiDAR-images with depth, intensity, and ambient radiation
channels. These measurements originate from the same sensor, rendering them
perfectly aligned in time and space. We demonstrate that such LiDAR-images are
sufficient for the real-car road-following task and perform at least equally to
camera-based models in the tested conditions, with the difference increasing
when needing to generalize to new weather conditions. In the second direction
of study, we reveal that the temporal smoothness of off-policy prediction
sequences correlates equally well with actual on-policy driving ability as the
commonly used mean absolute error.

本研究证明，在自动驾驶领域，结合深度传感和视觉信息对于现实汽车道路跟随任务相当有效，且具备时间和空间的完美对齐；同时，从预测效果上可以看出，离线预测序列的时间平滑程度能够等效于常用的平均绝对误差。

利用激光雷达作为摄像头进行端到端驾驶

LiDAR-as-Camera for End-to-End Driving

For human drivers, having rear and side-view mirrors is vital for safe
driving. They deliver a more complete view of what is happening around the car.
Human drivers also heavily exploit their mental map for navigation.
Nonetheless, several methods have been published that learn driving models with
only a front-facing camera and without a route planner. This lack of
information renders the self-driving task quite intractable. We investigate the
problem in a more realistic setting, which consists of a surround-view camera
system with eight cameras, a route planner, and a CAN bus reader. In
particular, we develop a sensor setup that provides data for a 360-degree view
of the area surrounding the vehicle, the driving route to the destination, and
low-level driving maneuvers (e.g. steering angle and speed) by human drivers.
With such a sensor setup we collect a new driving dataset, covering diverse
driving scenarios and varying weather/illumination conditions. Finally, we
learn a novel driving model by integrating information from the surround-view
cameras and the route planner. Two route planners are exploited: 1) by
representing the planned routes on OpenStreetMap as a stack of GPS coordinates,
and 2) by rendering the planned routes on TomTom Go Mobile and recording the
progression into a video. Our experiments show that: 1) 360-degree
surround-view cameras help avoid failures made with a single front-view camera,
in particular for city driving and intersection scenarios; and 2) route
planners help the driving task significantly, especially for steering angle
prediction.

通过一个由 8 个摄像头和路线规划器组成的组合实现自动驾驶， Collect driving data on various scenarios and weather conditions and learn a driving model by integrating information from the cameras and the route planner, which can significantly improve steering angle prediction.