AutoAugment has sparked an interest in automated augmentation methods for deep learning models. These methods estimate image transformation policies for train data that improve generalization to test data. While recent papers evolved in the direction of decreasing policy search complexity, we show that those methods are not robust when applied to biased and noisy data. To overcome these limitations, we reformulate AutoAugment as a generalized automated dataset optimization (AutoDO) task that minimizes the distribution shift between test data and distorted train dataset. In our AutoDO model, we explicitly estimate a set of per-point hyperparameters to flexibly change distribution of train data. In particular, we include hyperparameters for augmentation, loss weights, and soft-labels that are jointly estimated using implicit differentiation. We develop a theoretical probabilistic interpretation of this framework using Fisher information and show that its complexity scales linearly with the dataset size. Our experiments on SVHN, CIFAR-10/100, and ImageNet classification show up to 9.3% improvement for biased datasets with label noise compared to prior methods and, importantly, up to 36.6% gain for underrepresented SVHN classes.

AutoDO可用于深度学习模型的自动数据扩充，通过对损失权重、软标签等超参数进行联合估计来最小化测试数据和失真的训练数据之间的分布偏移，实现了偏差数据和标签噪音下的机器学习模型优化，相较于之前的方法，有着多达9.3%的准确度提升，并且在SVHN分类优化上达到了36.6%的增益。

AutoDO: 基于可扩展概率隐式微分的鲁棒性标准化数据自适应增强方法