Multi-view stereo reconstruction (MVS) in the wild requires to first estimate
the camera parameters e.g. intrinsic and extrinsic parameters. These are
usually tedious and cumbersome to obtain, yet they are mandatory to triangulate
corresponding pixels in 3D space, which is the core of all best performing MVS
algorithms. In this work, we take an opposite stance and introduce DUSt3R, a
radically novel paradigm for Dense and Unconstrained Stereo 3D Reconstruction
of arbitrary image collections, i.e. operating without prior information about
camera calibration nor viewpoint poses. We cast the pairwise reconstruction
problem as a regression of pointmaps, relaxing the hard constraints of usual
projective camera models. We show that this formulation smoothly unifies the
monocular and binocular reconstruction cases. In the case where more than two
images are provided, we further propose a simple yet effective global alignment
strategy that expresses all pairwise pointmaps in a common reference frame. We
base our network architecture on standard Transformer encoders and decoders,
allowing us to leverage powerful pretrained models. Our formulation directly
provides a 3D model of the scene as well as depth information, but
interestingly, we can seamlessly recover from it, pixel matches, relative and
absolute camera. Exhaustive experiments on all these tasks showcase that the
proposed DUSt3R can unify various 3D vision tasks and set new SoTAs on
monocular/multi-view depth estimation as well as relative pose estimation. In
summary, DUSt3R makes many geometric 3D vision tasks easy.

DUSt3R 是一种新颖的范式，用于进行任意图像集合的密集和无约束立体三维重建，通过将成对的重建问题作为点图的回归来实现，进而实现单眼和双眼重建的统一，提供场景的三维模型和深度信息，并能够轻松恢复像素匹配、相对和绝对相机，DUSt3R 能够简化多种几何三维视觉任务。

DUSt3R：简化几何三维视觉

DUSt3R: Geometric 3D Vision Made Easy

We present a self-supervised approach to training convolutional neural
networks for dense depth estimation from monocular endoscopy data without a
priori modeling of anatomy or shading. Our method only requires sequential data
from monocular endoscopic videos and a multi-view stereo reconstruction method,
e.g. structure from motion, that supervises learning in a sparse but accurate
manner. Consequently, our method requires neither manual interaction, such as
scaling or labeling, nor patient CT in the training and application phases. We
demonstrate the performance of our method on sinus endoscopy data from two
patients and validate depth prediction quantitatively using corresponding
patient CT scans where we found submillimeter residual errors.

本文提出一种无监督的自我监督方法，用于训练卷积神经网络从单目内窥镜数据中密集地估计深度，而无需先验建模解剖或着色。

单目内窥镜中用于密集深度估计的自监督学习

Self-supervised Learning for Dense Depth Estimation in Monocular  Endoscopy

We present DeepMVS, a deep convolutional neural network (ConvNet) for
multi-view stereo reconstruction. Taking an arbitrary number of posed images as
input, we first produce a set of plane-sweep volumes and use the proposed
DeepMVS network to predict high-quality disparity maps. The key contributions
that enable these results are (1) supervised pretraining on a photorealistic
synthetic dataset, (2) an effective method for aggregating information across a
set of unordered images, and (3) integrating multi-layer feature activations
from the pre-trained VGG-19 network. We validate the efficacy of DeepMVS using
the ETH3D Benchmark. Our results show that DeepMVS compares favorably against
state-of-the-art conventional MVS algorithms and other ConvNet based methods,
particularly for near-textureless regions and thin structures.

DeepMVS 是一种用于多视图立体重建的深度卷积神经网络 (ConvNet)，它可以对任意数量的姿态图像进行处理以预测高质量的视差图，其有效的信息聚合方法和采用 VGG-19 网络的多层特征激活实现了优异的效果。