The widespread use of mobile devices for all kind of transactions makes
necessary reliable and real-time identity authentication, leading to the
adoption of face recognition (FR) via the cameras embedded in such devices.
Progress of deep Convolutional Neural Networks (CNNs) has provided substantial
advances in FR. Nonetheless, the size of state-of-the-art architectures is
unsuitable for mobile deployment, since they often encompass hundreds of
megabytes and millions of parameters. We address this by studying methods for
deep network compression applied to FR. In particular, we apply network pruning
based on Taylor scores, where less important filters are removed iteratively.
The method is tested on three networks based on the small SqueezeNet (1.24M
parameters) and the popular MobileNetv2 (3.5M) and ResNet50 (23.5M)
architectures. These have been selected to showcase the method on CNNs with
different complexities and sizes. We observe that a substantial percentage of
filters can be removed with minimal performance loss. Also, filters with the
highest amount of output channels tend to be removed first, suggesting that
high-dimensional spaces within popular CNNs are over-dimensionated.

通过深度网络剪枝等方法，对基于小型 SqueezeNet、流行的 MobileNetv2 和 ResNet50 架构的网络进行压缩，以实现移动设备上的可靠且实时的人脸识别。

深度网络剪枝：关于人脸识别中 CNN 的比较研究

Deep Network Pruning: A Comparative Study on CNNs in Face Recognition

In recent years, deep network pruning has attracted significant attention in
order to enable the rapid deployment of AI into small devices with computation
and memory constraints. Pruning is often achieved by dropping redundant
weights, neurons, or layers of a deep network while attempting to retain a
comparable test performance. Many deep pruning algorithms have been proposed
with impressive empirical success. However, existing approaches lack a
quantifiable measure to estimate the compressibility of a sub-network during
each pruning iteration and thus may under-prune or over-prune the model. In
this work, we propose PQ Index (PQI) to measure the potential compressibility
of deep neural networks and use this to develop a Sparsity-informed Adaptive
Pruning (SAP) algorithm. Our extensive experiments corroborate the hypothesis
that for a generic pruning procedure, PQI decreases first when a large model is
being effectively regularized and then increases when its compressibility
reaches a limit that appears to correspond to the beginning of underfitting.
Subsequently, PQI decreases again when the model collapse and significant
deterioration in the performance of the model start to occur. Additionally, our
experiments demonstrate that the proposed adaptive pruning algorithm with
proper choice of hyper-parameters is superior to the iterative pruning
algorithms such as the lottery ticket-based pruning methods, in terms of both
compression efficiency and robustness.

本文提出了 PQ Index (PQI) 作为衡量深度神经网络可压缩性的量化指标，并基于此开发了一种基于稀疏性信息的自适应裁剪算法 (SAP)，该算法在压缩效率和稳健性方面优于遗传算法等迭代型裁剪算法。