We introduce the Dynamic Capacity Network (DCN), a neural network that can
adaptively assign its capacity across different portions of the input data.
This is achieved by combining modules of two types: low-capacity sub-networks
and high-capacity sub-networks. The low-capacity sub-networks are applied
across most of the input, but also provide a guide to select a few portions of
the input on which to apply the high-capacity sub-networks. The selection is
made using a novel gradient-based attention mechanism, that efficiently
identifies input regions for which the DCN's output is most sensitive and to
which we should devote more capacity. We focus our empirical evaluation on the
Cluttered MNIST and SVHN image datasets. Our findings indicate that DCNs are
able to drastically reduce the number of computations, compared to traditional
convolutional neural networks, while maintaining similar or even better
performance.

本文介绍了一种名为动态容量网络（DCN）的神经网络，可以通过结合低容量和高容量子网络并运用一种新颖的基于梯度的注意力机制自适应地在不同部分的输入数据中进行容量分配，实现对 Cluttered MNIST 和 SVHN 图像数据集的实证评估表明，相较于传统的卷积神经网络，DCN 可以显著降低计算数量，同时保持相似甚至更好的性能。