To operate safely, an automated vehicle (AV) must anticipate how the
environment around it will evolve. For that purpose, it is important to know
which prediction models are most appropriate for every situation. Currently,
assessment of prediction models is often performed over a set of trajectories
without distinction of the type of movement they capture, resulting in the
inability to determine the suitability of each model for different situations.
In this work we illustrate how standardized evaluation methods result in wrong
conclusions regarding a model's predictive capabilities, preventing a clear
assessment of prediction models and potentially leading to dangerous on-road
situations. We argue that following evaluation practices in safety assessment
for AVs, assessment of prediction models should be performed in a
scenario-based fashion. To encourage scenario-based assessment of prediction
models and illustrate the dangers of improper assessment, we categorize
trajectories of the Waymo Open Motion dataset according to the type of movement
they capture. Next, three different models are thoroughly evaluated for
different trajectory types and prediction horizons. Results show that common
evaluation methods are insufficient and the assessment should be performed
depending on the application in which the model will operate.

本文研究了自动驾驶车辆的安全性评估中的预测模型。作者指出，常用的评估方法不足以准确地评估预测模型，应该采用场景评估方法进行评估。通过 Waymo 开放运动数据集的轨迹分类，作者评估了三种不同的预测模型。结果表明，应根据模型所用于的应用程序的要求进行评估。