Facial expression editing has attracted increasing attention with the advance
of deep neural networks in recent years. However, most existing methods suffer
from compromised editing fidelity and limited usability as they either ignore
pose variations (unrealistic editing) or require paired training data (not easy
to collect) for pose controls. This paper presents POCE, an innovative
pose-controllable expression editing network that can generate realistic facial
expressions and head poses simultaneously with just unpaired training images.
POCE achieves the more accessible and realistic pose-controllable expression
editing by mapping face images into UV space, where facial expressions and head
poses can be disentangled and edited separately. POCE has two novel designs.
The first is self-supervised UV completion that allows to complete UV maps
sampled under different head poses, which often suffer from self-occlusions and
missing facial texture. The second is weakly-supervised UV editing that allows
to generate new facial expressions with minimal modification of facial
identity, where the synthesized expression could be controlled by either an
expression label or directly transplanted from a reference UV map via feature
transfer. Extensive experiments show that POCE can learn from unpaired face
images effectively, and the learned model can generate realistic and
high-fidelity facial expressions under various new poses.

该研究提出了一种新的姿态可控表情编辑网络 POCE，通过映射面部图像到 UV 空间，将面部表情和头部姿态分离，并可单独进行编辑，实现更易接近和真实的姿态可控表情编辑。

POCE: 姿态可控的表情编辑

POCE: Pose-Controllable Expression Editing

Estimating 3D mesh of the human body from a single 2D image is an important
task with many applications such as augmented reality and Human-Robot
interaction. However, prior works reconstructed 3D mesh from global image
feature extracted by using convolutional neural network (CNN), where the dense
correspondences between the mesh surface and the image pixels are missing,
leading to suboptimal solution. This paper proposes a model-free 3D human mesh
estimation framework, named DecoMR, which explicitly establishes the dense
correspondence between the mesh and the local image features in the UV space
(i.e. a 2D space used for texture mapping of 3D mesh). DecoMR first predicts
pixel-to-surface dense correspondence map (i.e., IUV image), with which we
transfer local features from the image space to the UV space. Then the
transferred local image features are processed in the UV space to regress a
location map, which is well aligned with transferred features. Finally we
reconstruct 3D human mesh from the regressed location map with a predefined
mapping function. We also observe that the existing discontinuous UV map are
unfriendly to the learning of network. Therefore, we propose a novel UV map
that maintains most of the neighboring relations on the original mesh surface.
Experiments demonstrate that our proposed local feature alignment and
continuous UV map outperforms existing 3D mesh based methods on multiple public
benchmarks. Code will be made available at
this https URL

本研究提出了一种无模型的 3D 人体网格估计框架 DecoMR，通过显式建立网格和局部图像特征在 UV 空间中的密集对应关系来重建人体 3D 网格，并设计了一种新的连续 UV 映射，优于现有的基于 3D 网格的方法。