Heterogeneous Face Recognition (HFR) refers to matching cross-domain faces
and plays a crucial role in public security. Nevertheless, HFR is confronted
with challenges from large domain discrepancy and insufficient heterogeneous
data. In this paper, we formulate HFR as a dual generation problem, and tackle
it via a novel Dual Variational Generation (DVG-Face) framework. Specifically,
a dual variational generator is elaborately designed to learn the joint
distribution of paired heterogeneous images. However, the small-scale paired
heterogeneous training data may limit the identity diversity of sampling. In
order to break through the limitation, we propose to integrate abundant
identity information of large-scale visible data into the joint distribution.
Furthermore, a pairwise identity preserving loss is imposed on the generated
paired heterogeneous images to ensure their identity consistency. As a
consequence, massive new diverse paired heterogeneous images with the same
identity can be generated from noises. The identity consistency and identity
diversity properties allow us to employ these generated images to train the HFR
network via a contrastive learning mechanism, yielding both domain-invariant
and discriminative embedding features. Concretely, the generated paired
heterogeneous images are regarded as positive pairs, and the images obtained
from different samplings are considered as negative pairs. Our method achieves
superior performances over state-of-the-art methods on seven challenging
databases belonging to five HFR tasks, including NIR-VIS, Sketch-Photo,
Profile-Frontal Photo, Thermal-VIS, and ID-Camera. The related code will be
released at this https URL

该研究提出了一种双重生成模型，通过引入正面样本和负面样本，从而生成大量多样的异构人脸图像以训练异构人脸识别（HFR）网络，使其具有域不变且具有区分度的嵌入特征，并在多项挑战性数据库的 5 个 HFR 任务上实现了优越的表现。

DVG-Face：双重变分生成用于异质人脸识别

DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition

Heterogeneous Face Recognition (HFR) is a challenging issue because of the
large domain discrepancy and a lack of heterogeneous data. This paper considers
HFR as a dual generation problem, and proposes a novel Dual Variational
Generation (DVG) framework. It generates large-scale new paired heterogeneous
images with the same identity from noise, for the sake of reducing the domain
gap of HFR. Specifically, we first introduce a dual variational autoencoder to
represent a joint distribution of paired heterogeneous images. Then, in order
to ensure the identity consistency of the generated paired heterogeneous
images, we impose a distribution alignment in the latent space and a pairwise
identity preserving in the image space. Moreover, the HFR network reduces the
domain discrepancy by constraining the pairwise feature distances between the
generated paired heterogeneous images. Extensive experiments on four HFR
databases show that our method can significantly improve state-of-the-art
results. The related code is available at this https URL

本文提出一种新的双变分生成框架 (DVG)，通过生成大规模的具有相同身份的异构图像，从噪声中减少异构人脸识别 (HFR) 中领域差距，实现了身份一致性和配对特征距离的约束，有效缩小了领域差距。经过在四个 HFR 数据集上的广泛实验测试，表明我们的方法可以显著改进最先进的结果。