Neural networks are usually over-parameterized with significant redundancy in
the number of required neurons which results in unnecessary computation and
memory usage at inference time. One common approach to address this issue is to
prune these big networks by removing extra neurons and parameters while
maintaining the accuracy. In this paper, we propose NoiseOut, a fully automated
pruning algorithm based on the correlation between activations of neurons in
the hidden layers. We prove that adding additional output neurons with entirely
random targets results into a higher correlation between neurons which makes
pruning by NoiseOut even more efficient. Finally, we test our method on various
networks and datasets. These experiments exhibit high pruning rates while
maintaining the accuracy of the original network.

本文提出了一种全自动剪枝算法 NoiseOut，该算法基于隐藏层神经元的激活相关性，并证明增加完全随机目标的输出神经元会导致神经元之间的更高相关性，从而使 NoiseOut 剪枝更加有效。通过在各种网络和数据集上进行实验，表明该方法具有高剪枝率，同时保持原始网络的精度。