Facial geometry and appearance capture have demonstrated tremendous success
in 3D scanning real humans in studios. Recent works propose to democratize this
technique while keeping the results high quality. However, they are still
inconvenient for daily usage. In addition, they focus on an easier problem of
only capturing facial skin. This paper proposes a novel method for high-quality
face capture, featuring an easy-to-use system and the capability to model the
complete face with skin, mouth interior, hair, and eyes. We reconstruct facial
geometry and appearance from a single co-located smartphone flashlight sequence
captured in a dim room where the flashlight is the dominant light source (e.g.
rooms with curtains or at night). To model the complete face, we propose a
novel hybrid representation to effectively model both eyes and other facial
regions, along with novel techniques to learn it from images. We apply a
combined lighting model to compactly represent real illuminations and exploit a
morphable face albedo model as a reflectance prior to disentangle diffuse and
specular. Experiments show that our method can capture high-quality 3D
relightable scans.

该论文提出了一种高质量的面部捕捉方法，利用单个手机闪光灯序列在昏暗的房间中捕捉面部几何和外观，模拟了完整的面部包括皮肤、口腔内部、头发和眼睛，并展示了可以捕捉高质量的 3D 可光照化扫描。

高质量家庭脸部几何和外观捕捉

High-Quality Facial Geometry and Appearance Capture at Home

The objective of this paper is to separate a target speaker's speech from a
mixture of two speakers using a deep audio-visual speech separation network.
Unlike previous works that used lip movement on video clips or pre-enrolled
speaker information as an auxiliary conditional feature, we use a single face
image of the target speaker. In this task, the conditional feature is obtained
from facial appearance in cross-modal biometric task, where audio and visual
identity representations are shared in latent space. Learnt identities from
facial images enforce the network to isolate matched speakers and extract the
voices from mixed speech. It solves the permutation problem caused by swapped
channel outputs, frequently occurred in speech separation tasks. The proposed
method is far more practical than video-based speech separation since user
profile images are readily available on many platforms. Also, unlike
speaker-aware separation methods, it is applicable on separation with unseen
speakers who have never been enrolled before. We show strong qualitative and
quantitative results on challenging real-world examples.

使用深度学习中的音频 - 视觉语音分离网络，从混音状态中分离出指定说话者的语音，且仅使用目标说话者的单一面部图像，通过潜在空间的交叉模态生物识别任务中的面部外观获得条件特征，可隔离和提取这些说话者的语音，为语音分离任务中的通道交换问题提供解决方案，同时也适用于未知说话者。

FaceFilter: 通过静态图像进行音视频语音分离

FaceFilter: Audio-visual speech separation using still images

Makeup is widely used to improve facial attractiveness and is well accepted
by the public. However, different makeup styles will result in significant
facial appearance changes. It remains a challenging problem to match makeup and
non-makeup face images. This paper proposes a learning from generation approach
for makeup-invariant face verification by introducing a bi-level adversarial
network (BLAN). To alleviate the negative effects from makeup, we first
generate non-makeup images from makeup ones, and then use the synthesized
non-makeup images for further verification. Two adversarial networks in BLAN
are integrated in an end-to-end deep network, with the one on pixel level for
reconstructing appealing facial images and the other on feature level for
preserving identity information. These two networks jointly reduce the sensing
gap between makeup and non-makeup images. Moreover, we make the generator well
constrained by incorporating multiple perceptual losses. Experimental results
on three benchmark makeup face datasets demonstrate that our method achieves
state-of-the-art verification accuracy across makeup status and can produce
photo-realistic non-makeup face images.

本文提出了一种使用双层对抗网络 (BLAN) 进行学习的方法，以解决化妆对面部识别的负面影响，首先从化妆样本中生成不化妆的图像，再进行进一步的验证，实验结果证明我们的方法能够达到业界最高的识别准确率，并且能够产生高质量的真实不化妆图像。