Recent learning approaches that implicitly represent surface geometry using
coordinate-based neural representations have shown impressive results in the
problem of multi-view 3D reconstruction. The effectiveness of these techniques
is, however, subject to the availability of a large number (several tens) of
input views of the scene, and computationally demanding optimizations. In this
paper, we tackle these limitations for the specific problem of few-shot full 3D
head reconstruction, by endowing coordinate-based representations with a
probabilistic shape prior that enables faster convergence and better
generalization when using few input images (down to three). First, we learn a
shape model of 3D heads from thousands of incomplete raw scans using implicit
representations. At test time, we jointly overfit two coordinate-based neural
networks to the scene, one modeling the geometry and another estimating the
surface radiance, using implicit differentiable rendering. We devise a
two-stage optimization strategy in which the learned prior is used to
initialize and constrain the geometry during an initial optimization phase.
Then, the prior is unfrozen and fine-tuned to the scene. By doing this, we
achieve high-fidelity head reconstructions, including hair and shoulders, and
with a high level of detail that consistently outperforms both state-of-the-art
3D Morphable Models methods in the few-shot scenario, and non-parametric
methods when large sets of views are available.

本研究介绍了一种利用概率形状先验的坐标神经表示方法，结合隐式可微渲染，通过两阶段优化过程，成功实现了多视角三维重建中几乎与 3D 可塑性模型方法、视角较多时的非参数方法相当的高保真度的少视角全三维头部重建。

H3D-Net：小样本高保真 3D 头部重建

H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction

3D shape reconstruction from a single image is a highly ill-posed problem.
Modern deep learning based systems try to solve this problem by learning an
end-to-end mapping from image to shape via a deep network. In this paper, we
aim to solve this problem via an online optimization framework inspired by
traditional methods. Our framework employs a deep autoencoder to learn a set of
latent codes of 3D object shapes, which are fitted by a probabilistic shape
prior using Gaussian Mixture Model (GMM). At inference, the shape and pose are
jointly optimized guided by both image cues and deep shape prior without
relying on an initialization from any trained deep nets. Surprisingly, our
method achieves comparable performance to state-of-the-art methods even without
training an end-to-end network, which shows a promising step in this direction.

本文提出了一种在线优化的网络结构，通过避免依赖于深层神经网络的初始化，使用了概率形状先验指导深层特征以及图像线索，实现了从单个图像中重建三维形状，且相较于现有最先进方法，得到了可比较的结果。