A reliable and accurate 3D tracking framework is essential for predicting
future locations of surrounding objects and planning the observer's actions in
numerous applications such as autonomous driving. We propose a framework that
can effectively associate moving objects over time and estimate their full 3D
bounding box information from a sequence of 2D images captured on a moving
platform. The object association leverages quasi-dense similarity learning to
identify objects in various poses and viewpoints with appearance cues only.
After initial 2D association, we further utilize 3D bounding boxes
depth-ordering heuristics for robust instance association and motion-based 3D
trajectory prediction for re-identification of occluded vehicles. In the end,
an LSTM-based object velocity learning module aggregates the long-term
trajectory information for more accurate motion extrapolation. Experiments on
our proposed simulation data and real-world benchmarks, including KITTI,
nuScenes, and Waymo datasets, show that our tracking framework offers robust
object association and tracking on urban-driving scenarios. On the Waymo Open
benchmark, we establish the first camera-only baseline in the 3D tracking and
3D detection challenges. Our quasi-dense 3D tracking pipeline achieves
impressive improvements on the nuScenes 3D tracking benchmark with near five
times tracking accuracy of the best vision-only submission among all published
methods. Our code, data and trained models are available at
this https URL

该研究提出了一种可靠和准确的基于 3D 跟踪的框架，可在众多应用中预测周围物体的未来位置和规划观察者的行动，通过使用拟密度相似性学习，基于 2D 图像，进行初始 2D 关联，并进一步利用 3D 边界框深度排序启发式来实现强健的实例关联和基于运动的 3D 轨迹预测，实验表明，在基准测试中，该框架在城市驾驶场景下具有稳健的物体关联和跟踪，可用于自动驾驶。