This paper presents our proposed approach that won the first prize at the
ICLR competition on Hardware Aware Efficient Training. The challenge is to
achieve the highest possible accuracy in an image classification task in less
than 10 minutes. The training is done on a small dataset of 5000 images picked
randomly from CIFAR-10 dataset. The evaluation is performed by the competition
organizers on a secret dataset with 1000 images of the same size. Our approach
includes applying a series of technique for improving the generalization of
ResNet-9 including: sharpness aware optimization, label smoothing, gradient
centralization, input patch whitening as well as metalearning based training.
Our experiments show that the ResNet-9 can achieve the accuracy of 88% while
trained only on a 10% subset of CIFAR-10 dataset in less than 10 minuets

本文提出了在 ICLR 的竞赛中获得第一名的硬件感知高效训练方法，该方法针对在不到 10 分钟内在图像分类任务中实现尽可能高的准确性的挑战。通过在一个小数据集上进行训练，并应用一系列技术来改进 ResNet-9 的泛化性能，我们的实验表明，ResNet-9 可以在不到 10 分钟的时间内只使用 CIFAR-10 数据集的 10% 子集达到 88% 的准确率。