Autonomous driving gained huge traction in recent years, due to its potential
to change the way we commute. Much effort has been put into trying to estimate
the state of a vehicle. Meanwhile, learning to forecast the state of a vehicle
ahead introduces new capabilities, such as predicting dangerous situations.
Moreover, forecasting brings new supervision opportunities by learning to
predict richer a context, expressed by multiple horizons. Intuitively, a video
stream originated from a front-facing camera is necessary because it encodes
information about the upcoming road. Besides, historical traces of the
vehicle's states give more context. In this paper, we tackle multi-horizon
forecasting of vehicle states by fusing the two modalities. We design and
experiment with 3 end-to-end architectures that exploit 3D convolutions for
visual features extraction and 1D convolutions for features extraction from
speed and steering angle traces. To demonstrate the effectiveness of our
method, we perform extensive experiments on two publicly available real-world
datasets, Comma2k19 and the Udacity challenge. We show that we are able to
forecast a vehicle's state to various horizons, while outperforming the current
state-of-the-art results on the related task of driving state estimation. We
examine the contribution of vision features, and find that a model fed with
vision features achieves an error that is 56.6% and 66.9% of the error of a
model that doesn't use those features, on the Udacity and Comma2k19 datasets
respectively.

本研究针对自动驾驶中的多视角预测车辆状态的问题，提出了一种多视角与历史轨迹相融合的预测框架，并通过使用 3D 卷积从视频中提取视觉特征以及通过 1D 卷积从车速和转向角度轨迹数据中提取特征，实现了在实际数据集上的车辆状态预测任务中的优越表现。

基于视觉引导的预测 —— 多时序预测的视觉上下文

Vision-Guided Forecasting -- Visual Context for Multi-Horizon Time  Series Forecasting

This article presents AutoRally, a 1$:$5 scale robotics testbed for
autonomous vehicle research. AutoRally is designed for robustness, ease of use,
and reproducibility, so that a team of two people with limited knowledge of
mechanical engineering, electrical engineering, and computer science can
construct and then operate the testbed to collect real world autonomous driving
data in whatever domain they wish to study. Complete documentation to construct
and operate the platform is available online along with tutorials, example
controllers, and a driving dataset collected at the Georgia Tech Autonomous
Racing Facility. Offline estimation algorithms are used to determine parameters
for physics-based dynamics models using an adaptive limited memory joint state
unscented Kalman filter. Online vehicle state estimation using a factor graph
optimization scheme and a convolutional neural network for semantic
segmentation of drivable surface are presented. All algorithms are tested with
real world data from the fleet of six AutoRally robots at the Georgia Tech
Autonomous Racing Facility tracks, and serve as a demonstration of the
robot$'$s capabilities.

AutoRally 是一个 1:5 比例的机器人测试平台，可用于自动驾驶汽车研究，包括使用离线估计算法和卷积神经网络进行车辆状态估计以及探索可驾驶表面，使用真实世界的数据从六辆 AutoRally 机器人提供的验证结果证明了机器人的能力。