The Multi-Prize Lottery Ticket Hypothesis posits that randomly initialized
neural networks contain several subnetworks that achieve comparable accuracy to
fully trained models of the same architecture. However, current methods require
that the network is sufficiently overparameterized. In this work, we propose a
modification to two state-of-the-art algorithms (Edge-Popup and Biprop) that
finds high-accuracy subnetworks with no additional storage cost or scaling. The
algorithm, Iterative Weight Recycling, identifies subsets of important weights
within a randomly initialized network for intra-layer reuse. Empirically we
show improvements on smaller network architectures and higher prune rates,
finding that model sparsity can be increased through the "recycling" of
existing weights. In addition to Iterative Weight Recycling, we complement the
Multi-Prize Lottery Ticket Hypothesis with a reciprocal finding: high-accuracy,
randomly initialized subnetwork's produce diverse masks, despite being
generated with the same hyperparameter's and pruning strategy. We explore the
landscapes of these masks, which show high variability.

该研究介绍了一种名为 Iterative Weight Recycling 的算法，该算法可在初始化的神经网络中识别重要权重的子集以供重复使用，可以提高模型稀疏度，此外还支持了多奖励彩票票据假设的互补结果，即高精度，随机初始化的子网络产生多样的掩码，展现了高度的变异性。