Recent isotropic networks, such as ConvMixer and vision transformers, have
found significant success across visual recognition tasks, matching or
outperforming non-isotropic convolutional neural networks (CNNs). Isotropic
architectures are particularly well-suited to cross-layer weight sharing, an
effective neural network compression technique. In this paper, we perform an
empirical evaluation on methods for sharing parameters in isotropic networks
(SPIN). We present a framework to formalize major weight sharing design
decisions and perform a comprehensive empirical evaluation of this design
space. Guided by our experimental results, we propose a weight sharing strategy
to generate a family of models with better overall efficiency, in terms of
FLOPs and parameters versus accuracy, compared to traditional scaling methods
alone, for example compressing ConvMixer by 1.9x while improving accuracy on
ImageNet. Finally, we perform a qualitative study to further understand the
behavior of weight sharing in isotropic architectures. The code is available at
this https URL

本研究旨在对等向网络 (SPIN) 进行经验性评价，提出了一种权值共享策略，并对权值共享的行为进行了定性研究。实验结果表明，该策略可以比传统缩放方法更有效地提高模型效率，例如在 ImageNet 数据集上将 ConvMixer 压缩 1.9x 并提高准确性