The cross-sensor gap is one of the challenges that have aroused much research
interests in Heterogeneous Face Recognition (HFR). Although recent methods have
attempted to fill the gap with deep generative networks, most of them suffer
from the inevitable misalignment between different face modalities. Instead of
imaging sensors, the misalignment primarily results from facial geometric
variations that are independent of the spectrum. Rather than building a
monolithic but complex structure, this paper proposes a Pose Aligned
Cross-spectral Hallucination (PACH) approach to disentangle the independent
factors and deal with them in individual stages. In the first stage, an
Unsupervised Face Alignment (UFA) module is designed to align the facial shapes
of the near-infrared (NIR) images with those of the visible (VIS) images in a
generative way, where UV maps are effectively utilized as the shape guidance.
Thus the task of the second stage becomes spectrum translation with aligned
paired data. We develop a Texture Prior Synthesis (TPS) module to achieve
complexion control and consequently generate more realistic VIS images than
existing methods. Experiments on three challenging NIR-VIS datasets verify the
effectiveness of our approach in producing visually appealing images and
achieving state-of-the-art performance in HFR.

本研究提出一种姿态对齐的跨光谱幻觉方法，通过生成方法对准红外和可见光人脸图像，再使用一种纹理融合模块进行光谱转换，从而实现异构人脸识别。在三个具有挑战性的数据集上进行实验，验证了该方法在视觉和性能方面的有效性。