In this paper, we propose a novel method for 3D scene and object
reconstruction from sparse multi-view images. Different from previous methods
that leverage extra information such as depth or generalizable features across
scenes, our approach leverages the scene properties embedded in the multi-view
inputs to create precise pseudo-labels for optimization without any prior
training. Specifically, we introduce a geometry-guided approach that improves
surface reconstruction accuracy from sparse views by leveraging spherical
harmonics to predict the novel radiance while holistically considering all
color observations for a point in the scene. Also, our pipeline exploits proxy
geometry and correctly handles the occlusion in generating the pseudo-labels of
radiance, which previous image-warping methods fail to avoid. Our method,
dubbed Ray Augmentation (RayAug), achieves superior results on DTU and Blender
datasets without requiring prior training, demonstrating its effectiveness in
addressing the problem of sparse view reconstruction. Our pipeline is flexible
and can be integrated into other implicit neural reconstruction methods for
sparse views.

本文提出了一种从稀疏多视角图像重建三维场景和对象的新方法，通过利用嵌入在多视角输入中的场景属性，创建精确的伪标签进行优化，而无需任何先验训练。我们的方法通过利用球谐函数来预测场景中点的新辐射，综合考虑所有颜色观察结果，从而提高稀疏视图的表面重建准确性。同时，我们的方法通过代理几何和正确处理遮挡来生成辐射的伪标签，避免了以往的图像变形方法的问题。我们的方法在 DTU 和 Blender 数据集上取得了优越的结果，而无需先前的训练，展示了其在解决稀疏视图重建问题方面的有效性。我们的流程灵活且可集成到其他稀疏视图的隐式神经重建方法中。

基于几何引导的稀疏视图神经表面重建的光线增强

Geometry-Guided Ray Augmentation for Neural Surface Reconstruction with  Sparse Views

We present a method that enables synthesizing novel views and novel poses of
arbitrary human performers from sparse multi-view images. A key ingredient of
our method is a hybrid appearance blending module that combines the advantages
of the implicit body NeRF representation and image-based rendering. Existing
generalizable human NeRF methods that are conditioned on the body model have
shown robustness against the geometric variation of arbitrary human performers.
Yet they often exhibit blurry results when generalized onto unseen identities.
Meanwhile, image-based rendering shows high-quality results when sufficient
observations are available, whereas it suffers artifacts in sparse-view
settings. We propose Neural Image-based Avatars (NIA) that exploits the best of
those two methods: to maintain robustness under new articulations and
self-occlusions while directly leveraging the available (sparse) source view
colors to preserve appearance details of new subject identities. Our hybrid
design outperforms recent methods on both in-domain identity generalization as
well as challenging cross-dataset generalization settings. Also, in terms of
the pose generalization, our method outperforms even the per-subject optimized
animatable NeRF methods. The video results are available at
this https URL

本文提出一种基于稀疏多视图图像的神经图像化头像生成方法，利用了近年来提出的体视图下的神经反射场（NeRF）模型及图像合成技术，提高了新姿势下头像的变形能力和形体形变下姿势生成的只优。