This paper seeks to answer the question: as the (near-) orthogonality of
weights is found to be a favorable property for training deep convolutional
neural networks, how can we enforce it in more effective and easy-to-use ways?
We develop novel orthogonality regularizations on training deep CNNs, utilizing
various advanced analytical tools such as mutual coherence and restricted
isometry property. These plug-and-play regularizations can be conveniently
incorporated into training almost any CNN without extra hassle. We then
benchmark their effects on state-of-the-art models: ResNet, WideResNet, and
ResNeXt, on several most popular computer vision datasets: CIFAR-10, CIFAR-100,
SVHN and ImageNet. We observe consistent performance gains after applying those
proposed regularizations, in terms of both the final accuracies achieved, and
faster and more stable convergences. We have made our codes and pre-trained
models publicly available:
this https URL

本文旨在探讨如何在训练深度卷积神经网络时实现更有效和易于使用的（近）正交性权重。我们利用各种先进的分析工具，如相互相干性和受限等距性质，开发新型正交性正则化方法，可方便地应用于训练几乎任何卷积神经网络。通过在几个流行的计算机视觉数据集上使用 ResNet、WideResNet 和 ResNeXt 等最先进模型的基准测试，我们观察到使用这些提议的正则化方法后，模型的准确率和收敛速度都有更稳定的提高。我们已经将我们的代码和预训练模型公开。