Neural 3D scene representations have shown great potential for 3D
reconstruction from 2D images. However, reconstructing real-world captures of
complex scenes still remains a challenge. Existing generic 3D reconstruction
methods often struggle to represent fine geometric details and do not
adequately model reflective surfaces of large-scale scenes. Techniques that
explicitly focus on reflective surfaces can model complex and detailed
reflections by exploiting better reflection parameterizations. However, we
observe that these methods are often not robust in real unbounded scenarios
where non-reflective as well as reflective components are present. In this
work, we propose UniSDF, a general purpose 3D reconstruction method that can
reconstruct large complex scenes with reflections. We investigate both
view-based as well as reflection-based color prediction parameterization
techniques and find that explicitly blending these representations in 3D space
enables reconstruction of surfaces that are more geometrically accurate,
especially for reflective surfaces. We further combine this representation with
a multi-resolution grid backbone that is trained in a coarse-to-fine manner,
enabling faster reconstructions than prior methods. Extensive experiments on
object-level datasets DTU, Shiny Blender as well as unbounded datasets Mip-NeRF
360 and Ref-NeRF real demonstrate that our method is able to robustly
reconstruct complex large-scale scenes with fine details and reflective
surfaces. Please see our project page at
this https URL

提出了 UniSDF，一个可以重建具有反射表面的大规模复杂场景的通用 3D 重建方法。通过在 3D 空间中明确混合视图和反射颜色预测参数化技术，使得重建的表面在几何上更准确，特别是对于反射表面。通过与多分辨率网格主干结合，该方法以粗糙到精细的方式进行训练，实现比以前的方法更快的重建速度。大量实验表明，该方法能够稳健地重建具有精细细节和反射表面的复杂大规模场景。

UniSDF：统一神经表示用于具有反射的复杂场景的高保真度 3D 重建

UniSDF: Unifying Neural Representations for High-Fidelity 3D  Reconstruction of Complex Scenes with Reflections

Our goal is to efficiently learn personalized animatable 3D head avatars from
videos that are geometrically accurate, realistic, relightable, and compatible
with current rendering systems. While 3D meshes enable efficient processing and
are highly portable, they lack realism in terms of shape and appearance. Neural
representations, on the other hand, are realistic but lack compatibility and
are slow to train and render. Our key insight is that it is possible to
efficiently learn high-fidelity 3D mesh representations via differentiable
rendering by exploiting highly-optimized methods from traditional computer
graphics and approximating some of the components with neural networks. To that
end, we introduce \moniker, a technique that enables the creation of animatable
and relightable mesh avatars from a single monocular video. First, we learn a
canonical geometry using a mesh representation, enabling efficient
differentiable rasterization and straightforward animation via learned
blendshapes and linear blend skinning weights. Second, we follow
physically-based rendering and factor observed colors into intrinsic albedo,
roughness, and a neural representation of the illumination, allowing the
learned avatars to be relit in novel scenes. Since our input videos are
captured on a single device with a narrow field of view, modeling the
surrounding environment light is non-trivial. Based on the split-sum
approximation for modeling specular reflections, we address this by
approximating the pre-filtered environment map with a multi-layer perceptron
(MLP) modulated by the surface roughness, eliminating the need to explicitly
model the light. We demonstrate that our mesh-based avatar formulation,
combined with learned deformation, material, and lighting MLPs, produces
avatars with high-quality geometry and appearance, while also being efficient
to train and render compared to existing approaches.

我们提出了一种从单一单目视频中学习可个性化动画的、具有几何精度、逼真性、可重新照明性并且与当前渲染系统兼容的 3D 头像人物模型的高效方法。

FLARE: 快速学习可动态和可照明的网格化角色

FLARE: Fast Learning of Animatable and Relightable Mesh Avatars

A novel method for robust estimation, called Graph-Cut RANSAC, GC-RANSAC in
short, is introduced. To separate inliers and outliers, it runs the graph-cut
algorithm in the local optimization (LO) step which is applied when a
so-far-the-best model is found. The proposed LO step is conceptually simple,
easy to implement, globally optimal and efficient. GC-RANSAC is shown
experimentally, both on synthesized tests and real image pairs, to be more
geometrically accurate than state-of-the-art methods on a range of problems,
e.g. line fitting, homography, affine transformation, fundamental and essential
matrix estimation. It runs in real-time for many problems at a speed
approximately equal to that of the less accurate alternatives (in milliseconds
on standard CPU).

本研究介绍了一种名为 Graph-Cut RANSAC 的鲁棒性估计新方法，通过在局部优化（LO）步骤中运行图割算法，以区分内点和外点，该方法是概念简单、易于实现、全局最优和高效的。实验结果表明，Graph-Cut RANSAC 在线拟合，单应性，仿射变换，基础和本质矩阵评估等多种问题上几何精度比现有技术更高。