Neural growth is the process of growing a small neural network to a large
network and has been utilized to accelerate the training of deep neural
networks. One crucial aspect of neural growth is determining the optimal growth
timing. However, few studies investigate this systematically. Our study reveals
that neural growth inherently exhibits a regularization effect, whose intensity
is influenced by the chosen policy for growth timing. While this regularization
effect may mitigate the overfitting risk of the model, it may lead to a notable
accuracy drop when the model underfits. Yet, current approaches have not
addressed this issue due to their lack of consideration of the regularization
effect from neural growth. Motivated by these findings, we propose an
under/over fitting risk-aware growth timing policy, which automatically adjusts
the growth timing informed by the level of potential under/overfitting risks to
address both risks. Comprehensive experiments conducted using CIFAR-10/100 and
ImageNet datasets show that the proposed policy achieves accuracy improvements
of up to 1.3% in models prone to underfitting while achieving similar
accuracies in models suffering from overfitting compared to the existing
methods.

神经生长是将小型神经网络扩展到大型网络的过程，其优化生长时机的研究相对较少。本研究揭示了神经生长固有地具有正则化效应，其强度受生长时机政策的影响。为解决欠拟合和过拟合风险，本文提出了一种基于风险感知的生长时机策略，在潜在的欠拟合和过拟合风险水平的指导下自动调整生长时机。在使用 CIFAR-10/100 和 ImageNet 数据集进行的综合实验中，所提出的策略在容易欠拟合的模型中实现了高达 1.3% 的准确率改善，同时相对于现有方法，在过拟合模型中实现了相似的准确率。