Recent advances in neural reconstruction using posed image sequences have
made remarkable progress. However, due to the lack of depth information,
existing volumetric-based techniques simply duplicate 2D image features of the
object surface along the entire camera ray. We contend this duplication
introduces noise in empty and occluded spaces, posing challenges for producing
high-quality 3D geometry. Drawing inspiration from traditional multi-view
stereo methods, we propose an end-to-end 3D neural reconstruction framework
CVRecon, designed to exploit the rich geometric embedding in the cost volumes
to facilitate 3D geometric feature learning. Furthermore, we present
Ray-contextual Compensated Cost Volume (RCCV), a novel 3D geometric feature
representation that encodes view-dependent information with improved integrity
and robustness. Through comprehensive experiments, we demonstrate that our
approach significantly improves the reconstruction quality in various metrics
and recovers clear fine details of the 3D geometries. Our extensive ablation
studies provide insights into the development of effective 3D geometric feature
learning schemes. Project page: this https URL

提出了一种基于多视角立体技术的神经重建框架 CVRecon，使用创新的 3D 几何特征表示方法 RCCV，可以显着提高三维几何重建质量和恢复细节。

CVRecon: 重新定义神经重建的 3D 几何特征学习

CVRecon: Rethinking 3D Geometric Feature Learning For Neural  Reconstruction

Depth estimation is solved as a regression or classification problem in
existing learning-based multi-view stereo methods. Although these two
representations have recently demonstrated their excellent performance, they
still have apparent shortcomings, e.g., regression methods tend to overfit due
to the indirect learning cost volume, and classification methods cannot
directly infer the exact depth due to its discrete prediction. In this paper,
we propose a novel representation, termed Unification, to unify the advantages
of regression and classification. It can directly constrain the cost volume
like classification methods, but also realize the sub-pixel depth prediction
like regression methods. To excavate the potential of unification, we design a
new loss function named Unified Focal Loss, which is more uniform and
reasonable to combat the challenge of sample imbalance. Combining these two
unburdened modules, we present a coarse-to-fine framework, that we call
UniMVSNet. The results of ranking first on both DTU and Tanks and Temples
benchmarks verify that our model not only performs the best but also has the
best generalization ability.

本文提出了一种新的深度图表示方式 ——Unification，将回归和分类学习的优势结合在一起，在设计了新的统一 Focal Loss 的基础上，构建了一个粗到细的 Coarse-to-fine framework，称为 UniMVSNet，实现了最好的深度估计效果和最佳的泛化能力。

多视角立体视觉深度估计的反思：一种统一的表示方法

Rethinking Depth Estimation for Multi-View Stereo: A Unified Representation

Accurate stereo depth estimation plays a critical role in various 3D tasks in
both indoor and outdoor environments. Recently, learning-based multi-view
stereo methods have demonstrated competitive performance with a limited number
of views. However, in challenging scenarios, especially when building
cross-view correspondences is hard, these methods still cannot produce
satisfying results. In this paper, we study how to leverage a normal estimation
model and the predicted normal maps to improve the depth quality. We couple the
learning of a multi-view normal estimation module and a multi-view depth
estimation module. In addition, we propose a novel consistency loss to train an
independent consistency module that refines the depths from depth/normal pairs.
We find that the joint learning can improve both the prediction of normal and
depth, and the accuracy & smoothness can be further improved by enforcing the
consistency. Experiments on MVS, SUN3D, RGBD, and Scenes11 demonstrate the
effectiveness of our method and state-of-the-art performance.

本文研究如何利用法线估计模型和预测的法线图提高深度质量，方法包括联合学习多视角法线估计和深度估计模块以及提出一种新的一致性损失来训练一个独立的一致性模块来细化深度 / 法线对。实验结果表明，该方法具有高精度、平滑等优点，在多个数据集上的表现均优于现有技术。