Deep neural networks with skip-connections, such as ResNet, show excellent performance in various image classification benchmarks. It is though observed that the initial motivation behind them - training deeper networks - does not actually hold true, and the benefits come from increased capacity, rather than from depth. Motivated by this, and inspired from ResNet, we propose a simple Dirac weight parameterization, which allows us to train very deep plain networks without skip-connections, and achieve nearly the same performance. This parameterization has a minor computational cost at training time and no cost at all at inference. We're able to achieve 95.5% accuracy on CIFAR-10 with 34-layer deep plain network, surpassing 1001-layer deep ResNet, and approaching Wide ResNet. Our parameterization also mostly eliminates the need of careful initialization in residual and non-residual networks. The code and models for our experiments are available at https://github.com/szagoruyko/diracnets

深度神经网络已经在各种图片分类基准测试中显示出优秀的性能，而ResNet的一个新的简单的Dirac参数化指标，在训练时间计算成本上具有较小的成本，在推理阶段几乎没有任何成本，同时比ResNet具有类似的性能，并减少残差网络以及非残差网络的初始化需要。

DiracNets: 不使用跳连连接训练非常深的神经网络