Accurate detection of 3D objects is a fundamental problem in computer vision
and has an enormous impact on autonomous cars, augmented/virtual reality and
many applications in robotics. In this work we present a novel fusion of neural
network based state-of-the-art 3D detector and visual semantic segmentation in
the context of autonomous driving. Additionally, we introduce
Scale-Rotation-Translation score (SRTs), a fast and highly parameterizable
evaluation metric for comparison of object detections, which speeds up our
inference time up to 20\% and halves training time. On top, we apply
state-of-the-art online multi target feature tracking on the object
measurements to further increase accuracy and robustness utilizing temporal
information. Our experiments on KITTI show that we achieve same results as
state-of-the-art in all related categories, while maintaining the performance
and accuracy trade-off and still run in real-time. Furthermore, our model is
the first one that fuses visual semantic with 3D object detection.

本文提出了一种融合神经网络、视觉语义分割、在自主驾驶场景下的 3D 目标检测方法，引入了一种名为 SRTs 的快速评估指标，并采用状态优化的多目标特征跟踪机制，实现了高精度和实时性的平衡，并在 KITTI 基准测试中验证了方法的有效性。同时，该方法首次将视觉语义与 3D 目标检测相融合。