Visible (VIS) to near infrared (NIR) face matching is a challenging problem
due to the significant domain discrepancy between the domains and a lack of
sufficient data for training cross-modal matching algorithms. Existing
approaches attempt to tackle this problem by either synthesizing visible faces
from NIR faces, extracting domain-invariant features from these modalities, or
projecting heterogeneous data onto a common latent space for cross-modal
matching. In this paper, we take a different approach in which we make use of
the Disentangled Variational Representation (DVR) for cross-modal matching.
First, we model a face representation with an intrinsic identity information
and its within-person variations. By exploring the disentangled latent variable
space, a variational lower bound is employed to optimize the approximate
posterior for NIR and VIS representations. Second, aiming at obtaining more
compact and discriminative disentangled latent space, we impose a minimization
of the identity information for the same subject and a relaxed correlation
alignment constraint between the NIR and VIS modality variations. An
alternative optimization scheme is proposed for the disentangled variational
representation part and the heterogeneous face recognition network part. The
mutual promotion between these two parts effectively reduces the NIR and VIS
domain discrepancy and alleviates over-fitting. Extensive experiments on three
challenging NIR-VIS heterogeneous face recognition databases demonstrate that
the proposed method achieves significant improvements over the state-of-the-art
methods.

本文提出了一种利用 Disentangled Variational Representation (DVR) 进行 NIR 和 VIS 模态匹配的方法，在生成具有内在身份信息和内部变化的脸部表示的基础上，通过探索解缠绕潜变量空间，使用变分下界对 NIR 和 VIS 表示进行优化，取得了较大的改进。

异构人脸识别的解耦变分表示

Disentangled Variational Representation for Heterogeneous Face  Recognition

Heterogeneous face recognition (HFR) aims to match facial images acquired
from different sensing modalities with mission-critical applications in
forensics, security and commercial sectors. However, HFR is a much more
challenging problem than traditional face recognition because of large
intra-class variations of heterogeneous face images and limited training
samples of cross-modality face image pairs. This paper proposes a novel
approach namely Wasserstein CNN (convolutional neural networks, or WCNN for
short) to learn invariant features between near-infrared and visual face images
(i.e. NIR-VIS face recognition). The low-level layers of WCNN are trained with
widely available face images in visual spectrum. The high-level layer is
divided into three parts, i.e., NIR layer, VIS layer and NIR-VIS shared layer.
The first two layers aims to learn modality-specific features and NIR-VIS
shared layer is designed to learn modality-invariant feature subspace.
Wasserstein distance is introduced into NIR-VIS shared layer to measure the
dissimilarity between heterogeneous feature distributions. So W-CNN learning
aims to achieve the minimization of Wasserstein distance between NIR
distribution and VIS distribution for invariant deep feature representation of
heterogeneous face images. To avoid the over-fitting problem on small-scale
heterogeneous face data, a correlation prior is introduced on the
fully-connected layers of WCNN network to reduce parameter space. This prior is
implemented by a low-rank constraint in an end-to-end network. The joint
formulation leads to an alternating minimization for deep feature
representation at training stage and an efficient computation for heterogeneous
data at testing stage. Extensive experiments on three challenging NIR-VIS face
recognition databases demonstrate the significant superiority of Wasserstein
CNN over state-of-the-art methods.

本文提出了一种新颖的方法：Wasserstein CNN，用于学习近红外和可见光面部图像之间对于解决异构人脸识别具有挑战性的问题具有不变性的特征。