Neural network ensembles have been effectively used to improve generalization
by combining the predictions of multiple independently trained models. However,
the growing scale and complexity of deep neural networks have led to these
methods becoming prohibitively expensive and time consuming to implement.
Low-cost ensemble methods have become increasingly important as they can
alleviate the need to train multiple models from scratch while retaining the
generalization benefits that traditional ensemble learning methods afford. This
dissertation introduces and formalizes a low-cost framework for constructing
Subnetwork Ensembles, where a collection of child networks are formed by
sampling, perturbing, and optimizing subnetworks from a trained parent model.
We explore several distinct methodologies for generating child networks and we
evaluate their efficacy through a variety of ablation studies and established
benchmarks. Our findings reveal that this approach can greatly improve training
efficiency, parametric utilization, and generalization performance while
minimizing computational cost. Subnetwork Ensembles offer a compelling
framework for exploring how we can build better systems by leveraging the
unrealized potential of deep neural networks.

本论文介绍了一种低成本框架用于构建子网络集合，通过从已训练的母模型中采样、扰动和优化子网络的方式形成子网络集合，研究发现该方法可以显著提高训练效率、参数利用以及泛化性能，同时最小化计算成本，通过利用深度神经网络的潜力来构建更好的系统。