We present an efficient multi-view stereo (MVS) network for 3D reconstruction
from multiview images. While previous learning based reconstruction approaches
performed quite well, most of them estimate depth maps at a fixed resolution
using plane sweep volumes with a fixed depth hypothesis at each plane, which
requires densely sampled planes for desired accuracy and therefore is difficult
to achieve high resolution depth maps. In this paper we introduce a
coarseto-fine depth inference strategy to achieve high resolution depth. This
strategy estimates the depth map at coarsest level, while the depth maps at
finer levels are considered as the upsampled depth map from previous level with
pixel-wise depth residual. Thus, we narrow the depth searching range with
priori information from previous level and construct new cost volumes from the
pixel-wise depth residual to perform depth map refinement. Then the final depth
map could be achieved iteratively since all the parameters are shared between
different levels. At each level, the self-attention layer is introduced to the
feature extraction block for capturing the long range dependencies for depth
inference task, and the cost volume is generated using similarity measurement
instead of the variance based methods used in previous work. Experiments were
conducted on both the DTU benchmark dataset and recently released BlendedMVS
dataset. The results demonstrated that our model could outperform most
state-of-the-arts (SOTA) methods. The codebase of this project is at
this https URL

提出一种高效的多视角立体（MVS）网络以实现多视角图像的三维重建和深度推断，其采用策略推断出粗到细的深度图，其中引入自注意力层与相似度测量来生成新的代价体以进行深度图细化，最终实验表明该模型优于大多数 SOTA 方法。

基于注意力感知的代价体积金字塔的多视角立体网络用于三维重建

Attention Aware Cost Volume Pyramid Based Multi-view Stereo Network for  3D Reconstruction

We introduce a novel learning-based method to reconstruct the high-quality
geometry and complex, spatially-varying BRDF of an arbitrary object from a
sparse set of only six images captured by wide-baseline cameras under
collocated point lighting. We first estimate per-view depth maps using a deep
multi-view stereo network; these depth maps are used to coarsely align the
different views. We propose a novel multi-view reflectance estimation network
architecture that is trained to pool features from these coarsely aligned
images and predict per-view spatially-varying diffuse albedo, surface normals,
specular roughness and specular albedo. We do this by jointly optimizing the
latent space of our multi-view reflectance network to minimize the photometric
error between images rendered with our predictions and the input images. While
previous state-of-the-art methods fail on such sparse acquisition setups, we
demonstrate, via extensive experiments on synthetic and real data, that our
method produces high-quality reconstructions that can be used to render
photorealistic images.

本研究提出了一种基于多视角立体网络、光谱反射率、深度学习的方法，旨在从六幅图像中重建物体的高质量几何结构和空间变化的 BRDF，通过最小化光度误差优化多视角反射率网络的潜在空间，成功地实现了从稀疏数据中重建出高质量的物体，进而渲染出真实的图像。