Recent works based on convolutional encoder-decoder architecture and 3DMM
parameterization have shown great potential for canonical view reconstruction
from a single input image. Conventional CNN architectures benefit from
exploiting the spatial correspondence between the input and output pixels.
However, in 3D face reconstruction, the spatial misalignment between the input
image (e.g. face) and the canonical/UV output makes the feature
encoding-decoding process quite challenging. In this paper, to tackle this
problem, we propose a new network architecture, namely the Affine Convolution
Networks, which enables CNN based approaches to handle spatially
non-corresponding input and output images and maintain high-fidelity quality
output at the same time. In our method, an affine transformation matrix is
learned from the affine convolution layer for each spatial location of the
feature maps. In addition, we represent 3D human heads in UV space with
multiple components, including diffuse maps for texture representation,
position maps for geometry representation, and light maps for recovering more
complex lighting conditions in the real world. All the components can be
trained without any manual annotations. Our method is parametric-free and can
generate high-quality UV maps at resolution of 512 x 512 pixels, while previous
approaches normally generate 256 x 256 pixels or smaller. Our code will be
released once the paper got accepted.

通过引入仿射卷积网络，本文提出了一种新的网络架构来解决三维人脸重建中输入图像和输出图像之间的空间不对应的问题，并生成 512×512 像素的高质量 UV 贴图。

高质量的基于仿射卷积网络的 3D 人脸重建

High-Quality 3D Face Reconstruction with Affine Convolutional Networks

Reenacting facial images is an important task that can find numerous
applications. We proposed IFaceUV, a fully differentiable pipeline that
properly combines 2D and 3D information to conduct the facial reenactment task.
The three-dimensional morphable face models (3DMMs) and corresponding UV maps
are utilized to intuitively control facial motions and textures, respectively.
Two-dimensional techniques based on 2D image warping is further required to
compensate for missing components of the 3DMMs such as backgrounds, ear, hair
and etc. In our pipeline, we first extract 3DMM parameters and corresponding UV
maps from source and target images. Then, initial UV maps are refined by the UV
map refinement network and it is rendered to the image with the motion
manipulated 3DMM parameters. In parallel, we warp the source image according to
the 2D flow field obtained from the 2D warping network. Rendered and warped
images are combined in the final editing network to generate the final
reenactment image. Additionally, we tested our model for the audio-driven
facial reenactment task. Extensive qualitative and quantitative experiments
illustrate the remarkable performance of our method compared to other
state-of-the-art methods.

IFaceUV 提出了一种全可微的技术流程，该流程涵盖了 2D 和 3D 信息的所有关键方面，从而为面部再现任务提供了一个效果显着的解决方案。