Perceiving 3D objects from monocular inputs is crucial for robotic systems,
given its economy compared to multi-sensor settings. It is notably difficult as
a single image can not provide any clues for predicting absolute depth values.
Motivated by binocular methods for 3D object detection, we take advantage of
the strong geometry structure provided by camera ego-motion for accurate object
depth estimation and detection. We first make a theoretical analysis on this
general two-view case and notice two challenges: 1) Cumulative errors from
multiple estimations that make the direct prediction intractable; 2) Inherent
dilemmas caused by static cameras and matching ambiguity. Accordingly, we
establish the stereo correspondence with a geometry-aware cost volume as the
alternative for depth estimation and further compensate it with monocular
understanding to address the second problem. Our framework, named Depth from
Motion (DfM), then uses the established geometry to lift 2D image features to
the 3D space and detects 3D objects thereon. We also present a pose-free DfM to
make it usable when the camera pose is unavailable. Our framework outperforms
state-of-the-art methods by a large margin on the KITTI benchmark. Detailed
quantitative and qualitative analyses also validate our theoretical
conclusions. The code will be released at
this https URL

通过研究基于单目输入的三维物体感知，提出了一种利用相机自运动提供的强几何结构进行精确物体深度估计和检测的方法，并建立了一个名为 “DfM” 的框架，成功地提高了 KITTI 基准数据集上的检测性能。

基于运动深度的单眼三维物体检测

Monocular 3D Object Detection with Depth from Motion

We propose a new multi-frame method for efficiently computing scene flow
(dense depth and optical flow) and camera ego-motion for a dynamic scene
observed from a moving stereo camera rig. Our technique also segments out
moving objects from the rigid scene. In our method, we first estimate the
disparity map and the 6-DOF camera motion using stereo matching and visual
odometry. We then identify regions inconsistent with the estimated camera
motion and compute per-pixel optical flow only at these regions. This flow
proposal is fused with the camera motion-based flow proposal using fusion moves
to obtain the final optical flow and motion segmentation. This unified
framework benefits all four tasks - stereo, optical flow, visual odometry and
motion segmentation leading to overall higher accuracy and efficiency. Our
method is currently ranked third on the KITTI 2015 scene flow benchmark.
Furthermore, our CPU implementation runs in 2-3 seconds per frame which is 1-3
orders of magnitude faster than the top six methods. We also report a thorough
evaluation on challenging Sintel sequences with fast camera and object motion,
where our method consistently outperforms OSF [Menze and Geiger, 2015], which
is currently ranked second on the KITTI benchmark.

本研究提出了一种新的多帧场景流计算方法，包括景深和光流以及相机自运动，同时从移动立体相机中观察动态场景，并将移动对象与固定场景分割开来。该技术在立体匹配和视觉里程计中使用独特的策略，融合移动目标的光流信息和相机基于运动流的信息来提高场景深度和相机自运动估计的精度和效率。