Is there an initialization for deep networks that requires no learning?
ExplainFix adopts two design principles: the "fixed filters" principle that all
spatial filter weights of convolutional neural networks can be fixed at
initialization and never learned, and the "nimbleness" principle that only few
network parameters suffice. We contribute (a) visual model-based explanations,
(b) speed and accuracy gains, and (c) novel tools for deep convolutional neural
networks. ExplainFix gives key insights that spatially fixed networks should
have a steered initialization, that spatial convolution layers tend to
prioritize low frequencies, and that most network parameters are not necessary
in spatially fixed models. ExplainFix models have up to 100x fewer spatial
filter kernels than fully learned models and matching or improved accuracy. Our
extensive empirical analysis confirms that ExplainFix guarantees nimbler models
(train up to 17\% faster with channel pruning), matching or improved predictive
performance (spanning 13 distinct baseline models, four architectures and two
medical image datasets), improved robustness to larger learning rate, and
robustness to varying model size. We are first to demonstrate that all spatial
filters in state-of-the-art convolutional deep networks can be fixed at
initialization, not learned.

ExplainFix 提出了 “固定过滤器” 和 “灵活性” 两个设计原则，其模型设计可将所有卷积神经网络的空间过滤器权重在初始化时固定并且不再进行学习，而且仅需很少的网络参数。经过广泛的实证分析，ExplainFix 可提高模型的训练速度，匹配或优化预测性能，并且对变化的模型尺寸具有更强的鲁棒性。