A family of super deep networks, referred to as residual networks or ResNet,
achieved record-beating performance in various visual tasks such as image
recognition, object detection, and semantic segmentation. The ability to train
very deep networks naturally pushed the researchers to use enormous resources
to achieve the best performance. Consequently, in many applications super deep
residual networks were employed for just a marginal improvement in performance.
In this paper, we propose epsilon-ResNet that allows us to automatically
discard redundant layers, which produces responses that are smaller than a
threshold epsilon, with a marginal or no loss in performance. The
epsilon-ResNet architecture can be achieved using a few additional rectified
linear units in the original ResNet. Our method does not use any additional
variables nor numerous trials like other hyper-parameter optimization
techniques. The layer selection is achieved using a single training process and
the evaluation is performed on CIFAR-10, CIFAR-100, SVHN, and ImageNet
datasets. In some instances, we achieve about 80% reduction in the number of
parameters.

本文提出一种名为 epsilon-ResNet 的深度学习网络，通过单次训练中的层选取达到在图像识别等多个领域中达到与超深残差网络相当甚至更好的性能，同时减少了 80% 左右的参数数量。