Deep neural networks (DNN) have achieved remarkable success in various
fields, including computer vision and natural language processing. However,
training an effective DNN model still poses challenges. This paper aims to
propose a method to optimize the training effectiveness of DNN, with the goal
of improving model performance. Firstly, based on the observation that the DNN
parameters change in certain laws during training process, the potential of
parameter prediction for improving model training efficiency and performance is
discovered. Secondly, considering the magnitude of DNN model parameters,
hardware limitations and characteristics of Stochastic Gradient Descent (SGD)
for noise tolerance, a Parameter Linear Prediction (PLP) method is exploit to
perform DNN parameter prediction. Finally, validations are carried out on some
representative backbones. Experiment results show that compare to the normal
training ways, under the same training conditions and epochs, by employing
proposed PLP method, the optimal model is able to obtain average about 1%
accuracy improvement and 0.01 top-1/top-5 error reduction for Vgg16, Resnet18
and GoogLeNet based on CIFAR-100 dataset, which shown the effectiveness of the
proposed method on different DNN structures, and validated its capacity in
enhancing DNN training efficiency and performance.

提出了一种优化深度神经网络训练效果的方法，通过参数预测来改进模型性能，通过实验证实了该方法在不同的深度神经网络结构上提高了训练效率和性能。