3D scene reconstruction is a long-standing vision task. Existing approaches
can be categorized into geometry-based and learning-based methods. The former
leverages multi-view geometry but can face catastrophic failures due to the
reliance on accurate pixel correspondence across views. The latter was
proffered to mitigate these issues by learning 2D or 3D representation
directly. However, without a large-scale video or 3D training data, it can
hardly generalize to diverse real-world scenarios due to the presence of tens
of millions or even billions of optimization parameters in the deep network.
Recently, robust monocular depth estimation models trained with large-scale
datasets have been proven to possess weak 3D geometry prior, but they are
insufficient for reconstruction due to the unknown camera parameters, the
affine-invariant property, and inter-frame inconsistency. Here, we propose a
novel test-time optimization approach that can transfer the robustness of
affine-invariant depth models such as LeReS to challenging diverse scenes while
ensuring inter-frame consistency, with only dozens of parameters to optimize
per video frame. Specifically, our approach involves freezing the pre-trained
affine-invariant depth model's depth predictions, rectifying them by optimizing
the unknown scale-shift values with a geometric consistency alignment module,
and employing the resulting scale-consistent depth maps to robustly obtain
camera poses and achieve dense scene reconstruction, even in low-texture
regions. Experiments show that our method achieves state-of-the-art
cross-dataset reconstruction on five zero-shot testing datasets.

我们提出了一种新的测试时间优化方法，可以将仿射不变深度模型的健壮性转移到具有挑战性的多样化场景，同时确保帧间一致性，每个视频帧只需优化几十个参数。实验证明，我们的方法在五个零样本测试数据集上实现了最先进的跨数据集重建。

FrozenRecon：基于冻结深度模型的无姿态三维场景重建

FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models

In this work we present a monocular visual odometry (VO) algorithm which
leverages geometry-based methods and deep learning. Most existing VO/SLAM
systems with superior performance are based on geometry and have to be
carefully designed for different application scenarios. Moreover, most
monocular systems suffer from scale-drift issue.Some recent deep learning works
learn VO in an end-to-end manner but the performance of these deep systems is
still not comparable to geometry-based methods. In this work, we revisit the
basics of VO and explore the right way for integrating deep learning with
epipolar geometry and Perspective-n-Point (PnP) method. Specifically, we train
two convolutional neural networks (CNNs) for estimating single-view depths and
two-view optical flows as intermediate outputs. With the deep predictions, we
design a simple but robust frame-to-frame VO algorithm (DF-VO) which
outperforms pure deep learning-based and geometry-based methods. More
importantly, our system does not suffer from the scale-drift issue being aided
by a scale consistent single-view depth CNN. Extensive experiments on KITTI
dataset shows the robustness of our system and a detailed ablation study shows
the effect of different factors in our system.

本文提出了一种单目视觉里程计算法，它结合了基于几何方法和深度学习的思想，并利用两个卷积神经网络进行深度估计和光流估计。在 KITTI 数据集上的实验表明，该算法具有较强的鲁棒性和良好的性能。