Typical Convolutional Neural Networks (ConvNets) depend heavily on large
amounts of image data and resort to an iterative optimization algorithm (e.g.,
SGD or Adam) to learn network parameters, which makes training very time- and
resource-intensive. In this paper, we propose a new training paradigm and
formulate the parameter learning of ConvNets into a prediction task: given a
ConvNet architecture, we observe there exists correlations between image
datasets and their corresponding optimal network parameters, and explore if we
can learn a hyper-mapping between them to capture the relations, such that we
can directly predict the parameters of the network for an image dataset never
seen during the training phase. To do this, we put forward a new hypernetwork
based model, called PudNet, which intends to learn a mapping between datasets
and their corresponding network parameters, and then predicts parameters for
unseen data with only a single forward propagation. Moreover, our model
benefits from a series of adaptive hyper recurrent units sharing weights to
capture the dependencies of parameters among different network layers.
Extensive experiments demonstrate that our proposed method achieves good
efficacy for unseen image datasets on two kinds of settings: Intra-dataset
prediction and Inter-dataset prediction. Our PudNet can also well scale up to
large-scale datasets, e.g., ImageNet-1K. It takes 8967 GPU seconds to train
ResNet-18 on the ImageNet-1K using GC from scratch and obtain a top-5 accuracy
of 44.65 %. However, our PudNet costs only 3.89 GPU seconds to predict the
network parameters of ResNet-18 achieving comparable performance (44.92 %),
more than 2,300 times faster than the traditional training paradigm.

传统的卷积神经网络依赖于大量的图像数据，并使用迭代优化算法来学习网络参数，这使得训练非常耗时和资源密集。本文提出了一种新的训练范式，将卷积神经网络的参数学习形式化为一种预测任务，通过学习数据集与网络参数之间的超映射关系，来直接预测在训练阶段从未见过的图像数据的网络参数。这种基于超网络的模型称为 PudNet，通过共享权重的自适应超循环单元捕捉不同网络层之间的参数依赖关系。实验证明，我们提出的方法在两种设置下（数据集内预测和数据集间预测）对于未见过的图像数据集具有良好的效果。PudNet 还可以很好地应用于大规模数据集，例如 ImageNet-1K。训练 ResNet-18 使用 GC 从头开始，在 ImageNet-1K 上花费 8967 个 GPU 秒，并获得 44.65% 的前 5 准确率。然而，我们的 PudNet 仅花费 3.89 个 GPU 秒来预测 ResNet-18 的网络参数，达到相当的性能（44.92%），比传统训练范式快了超过 2300 倍。