In this paper, we present a novel method for dynamically expanding
Convolutional Neural Networks (CNNs) during training, aimed at meeting the
increasing demand for efficient and sustainable deep learning models. Our
approach, drawing from the seminal work on Self-Expanding Neural Networks
(SENN), employs a natural expansion score as an expansion criteria to address
the common issue of over-parameterization in deep convolutional neural
networks, thereby ensuring that the model's complexity is finely tuned to the
task's specific needs. A significant benefit of this method is its eco-friendly
nature, as it obviates the necessity of training multiple models of different
sizes. We employ a strategy where a single model is dynamically expanded,
facilitating the extraction of checkpoints at various complexity levels,
effectively reducing computational resource use and energy consumption while
also expediting the development cycle by offering diverse model complexities
from a single training session. We evaluate our method on the CIFAR-10 dataset
and our experimental results validate this approach, demonstrating that
dynamically adding layers not only maintains but also improves CNN performance,
underscoring the effectiveness of our expansion criteria. This approach marks a
considerable advancement in developing adaptive, scalable, and environmentally
considerate neural network architectures, addressing key challenges in the
field of deep learning.

本文介绍了一种动态扩展卷积神经网络的新方法，通过使用自适应扩展评分作为扩展标准，解决了深度卷积神经网络中参数过多的常见问题，从而确保模型的复杂性与任务需求相匹配。该方法的一个显著优势是环保性质，因为它省去了训练多个不同大小模型的必要性。通过单个模型的动态扩展，实现了从单个训练会话中提取具有不同复杂性级别的检查点，有效降低了计算资源使用和能源消耗，同时加快了开发周期，提供了多样化的模型复杂性。在 CIFAR-10 数据集上评估了该方法，实验结果验证了该方法的有效性，表明动态添加层不仅可以维持 CNN 性能，还可以改进其性能，从而解决了深度学习领域的关键挑战。