We present a deep learning pipeline that leverages network self-prior to
recover a full 3D model consisting of both a triangular mesh and a texture map
from the colored 3D point cloud. Different from previous methods either
exploiting 2D self-prior for image editing or 3D self-prior for pure surface
reconstruction, we propose to exploit a novel hybrid 2D-3D self-prior in deep
neural networks to significantly improve the geometry quality and produce a
high-resolution texture map, which is typically missing from the output of
commodity-level 3D scanners. In particular, we first generate an initial mesh
using a 3D convolutional neural network with 3D self-prior, and then encode
both 3D information and color information in the 2D UV atlas, which is further
refined by 2D convolutional neural networks with the self-prior. In this way,
both 2D and 3D self-priors are utilized for the mesh and texture recovery.
Experiments show that, without the need of any additional training data, our
method recovers the 3D textured mesh model of high quality from sparse input,
and outperforms the state-of-the-art methods in terms of both the geometry and
texture quality.

提出一种利用深度神经网络的全新 2D-3D 自先验策略的深度学习流程来重构含有三角网格和纹理贴图的完整 3D 模型，实验证明该方法不仅无需额外的训练数据即可从稀疏输入中恢复出高质量的 3D 纹理网格模型，而且在几何和纹理质量上均优于现有方法。