The objective of this paper is to learn a compact representation of image
sets for template-based face recognition. We make the following contributions:
first, we propose a network architecture which aggregates and embeds the face
descriptors produced by deep convolutional neural networks into a compact
fixed-length representation. This compact representation requires minimal
memory storage and enables efficient similarity computation. Second, we propose
a novel GhostVLAD layer that includes {\em ghost clusters}, that do not
contribute to the aggregation. We show that a quality weighting on the input
faces emerges automatically such that informative images contribute more than
those with low quality, and that the ghost clusters enhance the network's
ability to deal with poor quality images. Third, we explore how input feature
dimension, number of clusters and different training techniques affect the
recognition performance. Given this analysis, we train a network that far
exceeds the state-of-the-art on the IJB-B face recognition dataset. This is
currently one of the most challenging public benchmarks, and we surpass the
state-of-the-art on both the identification and verification protocols.

本文旨在针对模板基础人脸识别问题，提出了一种图像集的紧凑表示方法；基于深度卷积神经网络产生的人脸描述符，该方法利用网络架构进行聚合嵌入，生成了一种紧凑且固定长度的表达形式，该表达形式需要最小的内存存储并能够进行有效的相似性计算；同时还提出了包含 “幽灵簇” 的新型 GhostVLAD 层，该层能够使输入的高质量图像比低质量图像更具信息量，增强了网络处理低质量图像的能力；最后，我们在 IJB-B 人脸识别数据集上训练的模型比目前的最先进模型性能更高，且在身份识别和验证协议上均超过了先前的最高水平。

基于集合的人脸识别中的 GhostVLAD

GhostVLAD for set-based face recognition

Although deep learning has yielded impressive performance for face
recognition, many studies have shown that different networks learn different
feature maps: while some networks are more receptive to pose and illumination
others appear to capture more local information. Thus, in this work, we propose
a deep heterogeneous feature fusion network to exploit the complementary
information present in features generated by different deep convolutional
neural networks (DCNNs) for template-based face recognition, where a template
refers to a set of still face images or video frames from different sources
which introduces more blur, pose, illumination and other variations than
traditional face datasets. The proposed approach efficiently fuses the
discriminative information of different deep features by 1) jointly learning
the non-linear high-dimensional projection of the deep features and 2)
generating a more discriminative template representation which preserves the
inherent geometry of the deep features in the feature space. Experimental
results on the IARPA Janus Challenge Set 3 (Janus CS3) dataset demonstrate that
the proposed method can effectively improve the recognition performance. In
addition, we also present a series of covariate experiments on the face
verification task for in-depth qualitative evaluations for the proposed
approach.

本文提出了一种深度异构特征融合网络，用于增强不同深度卷积神经网络生成的特征之间的关联性，以实现基于模板的人脸识别任务，并在 IARPA Janus Challenge Set 3 (Janus CS3) 数据集上证明了本方法的有效性。