Motion forecasting is crucial in autonomous driving systems to anticipate the
future trajectories of surrounding agents such as pedestrians, vehicles, and
traffic signals. In end-to-end forecasting, the model must jointly detect from
sensor data (cameras or LiDARs) the position and past trajectories of the
different elements of the scene and predict their future location. We depart
from the current trend of tackling this task via end-to-end training from
perception to forecasting and we use a modular approach instead. Following a
recent study, we individually build and train detection, tracking, and
forecasting modules. We then only use consecutive finetuning steps to integrate
the modules better and alleviate compounding errors. Our study reveals that
this simple yet effective approach significantly improves performance on the
end-to-end forecasting benchmark. Consequently, our solution ranks first in the
Argoverse 2 end-to-end Forecasting Challenge held at CVPR 2024 Workshop on
Autonomous Driving (WAD), with 63.82 mAPf. We surpass forecasting results by
+17.1 points over last year's winner and by +13.3 points over this year's
runner-up. This remarkable performance in forecasting can be explained by our
modular paradigm, which integrates finetuning strategies and significantly
outperforms the end-to-end-trained counterparts.

使用模块化方法，将感知、跟踪和预测模块单独建立并训练，然后通过连续的微调步骤整合这些模块以改善性能，在 Argoverse 2 的终端预测挑战中取得了显著的性能提升。

Valeo4Cast：一种模块化的端到端预测方法

Valeo4Cast: A Modular Approach to End-to-End Forecasting

This study explores the application of self-supervised learning (SSL) to the
task of motion forecasting, an area that has not yet been extensively
investigated despite the widespread success of SSL in computer vision and
natural language processing. To address this gap, we introduce Forecast-MAE, an
extension of the mask autoencoders framework that is specifically designed for
self-supervised learning of the motion forecasting task. Our approach includes
a novel masking strategy that leverages the strong interconnections between
agents' trajectories and road networks, involving complementary masking of
agents' future or history trajectories and random masking of lane segments. Our
experiments on the challenging Argoverse 2 motion forecasting benchmark show
that Forecast-MAE, which utilizes standard Transformer blocks with minimal
inductive bias, achieves competitive performance compared to state-of-the-art
methods that rely on supervised learning and sophisticated designs. Moreover,
it outperforms the previous self-supervised learning method by a significant
margin. Code is available at this https URL

通过引入 Forecast-MAE，一种专为自我监督学习运动预测任务设计的掩模自编码器框架的扩展，利用标准 Transformer 块以及最小的内在偏差，我们在具有挑战性的 Argoverse 2 运动预测基准测试上进行的实验表明，Forecast-MAE 取得了与依赖于监督学习和复杂设计的最先进方法竞争性的性能，并且明显优于以前的自我监督学习方法。

预测 MAE：基于掩码自编码器的运动预测的自监督预训练

Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with  Masked Autoencoders

We introduce Argoverse 2 (AV2) - a collection of three datasets for
perception and forecasting research in the self-driving domain. The annotated
Sensor Dataset contains 1,000 sequences of multimodal data, encompassing
high-resolution imagery from seven ring cameras, and two stereo cameras in
addition to lidar point clouds, and 6-DOF map-aligned pose. Sequences contain
3D cuboid annotations for 26 object categories, all of which are
sufficiently-sampled to support training and evaluation of 3D perception
models. The Lidar Dataset contains 20,000 sequences of unlabeled lidar point
clouds and map-aligned pose. This dataset is the largest ever collection of
lidar sensor data and supports self-supervised learning and the emerging task
of point cloud forecasting. Finally, the Motion Forecasting Dataset contains
250,000 scenarios mined for interesting and challenging interactions between
the autonomous vehicle and other actors in each local scene. Models are tasked
with the prediction of future motion for "scored actors" in each scenario and
are provided with track histories that capture object location, heading,
velocity, and category. In all three datasets, each scenario contains its own
HD Map with 3D lane and crosswalk geometry - sourced from data captured in six
distinct cities. We believe these datasets will support new and existing
machine learning research problems in ways that existing datasets do not. All
datasets are released under the CC BY-NC-SA 4.0 license.

Argoverse 2 是一套适用于自动驾驶领域的三个数据集，包含多模态数据，如高分辨率图像，种类繁多的 3D 标注，未标注的 lidar 点云等，为 3D 感知模型的训练和评估提供了充足的数据和场景，并提供了自我监督和云点预测等新的数据学习任务。