We present an approach for estimating surface normals from in-the-wild color
images of faces. While data-driven strategies have been proposed for single
face images, limited available ground truth data makes this problem difficult.
To alleviate this issue, we propose a method that can leverage all available
image and normal data, whether paired or not, thanks to a novel cross-modal
learning architecture. In particular, we enable additional training with single
modality data, either color or normal, by using two encoder-decoder networks
with a shared latent space. The proposed architecture also enables face details
to be transferred between the image and normal domains, given paired data,
through skip connections between the image encoder and normal decoder. Core to
our approach is a novel module that we call deactivable skip connections, which
allows integrating both the auto-encoded and image-to-normal branches within
the same architecture that can be trained end-to-end. This allows learning of a
rich latent space that can accurately capture the normal information. We
compare against state-of-the-art methods and show that our approach can achieve
significant improvements, both quantitative and qualitative, with natural face
images.

通过交叉模态学习实现，使用共享空间的编码器 - 解码器网络来对面部图像中的表面法向进行预测，并通过跳跃连接实现面部细节在图像和法线领域之间的转移。使用此方法，可以在没有匹配数据的情况下使用单模态数据进行附加训练，学习了丰富的潜在空间后，能够大幅提高表面法线预测效果。