Near-infrared to visible (NIR-VIS) face recognition is the most common case
in heterogeneous face recognition, which aims to match a pair of face images
captured from two different modalities. Existing deep learning based methods
have made remarkable progress in NIR-VIS face recognition, while it encounters
certain newly-emerged difficulties during the pandemic of COVID-19, since
people are supposed to wear facial masks to cut off the spread of the virus. We
define this task as NIR-VIS masked face recognition, and find it problematic
with the masked face in the NIR probe image. First, the lack of masked face
data is a challenging issue for the network training. Second, most of the
facial parts (cheeks, mouth, nose etc.) are fully occluded by the mask, which
leads to a large amount of loss of information. Third, the domain gap still
exists in the remaining facial parts. In such scenario, the existing methods
suffer from significant performance degradation caused by the above issues. In
this paper, we aim to address the challenge of NIR-VIS masked face recognition
from the perspectives of training data and training method. Specifically, we
propose a novel heterogeneous training method to maximize the mutual
information shared by the face representation of two domains with the help of
semi-siamese networks. In addition, a 3D face reconstruction based approach is
employed to synthesize masked face from the existing NIR image. Resorting to
these practices, our solution provides the domain-invariant face representation
which is also robust to the mask occlusion. Extensive experiments on three
NIR-VIS face datasets demonstrate the effectiveness and
cross-dataset-generalization capacity of our method.

本论文提出一种新的异构培训方法和 3D 面部重建方法，用于最大化两个域的面部表示共享的相互信息，从而提供领域不变的面部表示，并且具有掩蔽的稳健性。