Directly learning multiple 3D objects motion from sequential images is
difficult, while the geometric bundle adjustment lacks the ability to localize
the invisible object centroid. To benefit from both the powerful object
understanding skill from deep neural network meanwhile tackle precise geometry
modeling for consistent trajectory estimation, we propose a joint
spatial-temporal optimization-based stereo 3D object tracking method. From the
network, we detect corresponding 2D bounding boxes on adjacent images and
regress an initial 3D bounding box. Dense object cues (local depth and local
coordinates) that associating to the object centroid are then predicted using a
region-based network. Considering both the instant localization accuracy and
motion consistency, our optimization models the relations between the object
centroid and observed cues into a joint spatial-temporal error function. All
historic cues will be summarized to contribute to the current estimation by a
per-frame marginalization strategy without repeated computation. Quantitative
evaluation on the KITTI tracking dataset shows our approach outperforms
previous image-based 3D tracking methods by significant margins. We also report
extensive results on multiple categories and larger datasets (KITTI raw and
Argoverse Tracking) for future benchmarking.

通过深度神经网络强大的目标理解，同时处理精确几何建模的一致轨迹估计，提出了一种基于联合时空优化的立体 3D 对象跟踪方法，在 KITTI 跟踪数据集上取得显著优于先前方法的定量评估结果，并在多个类别和大型数据集（KITTI 原始和 Argoverse 跟踪）上进行了广泛的结果报告以供未来基准测试。