Adapting to the structure of data distributions (such as symmetry and
transformation invariances) is an important challenge in machine learning.
Invariances can be built into the learning process by architecture design, or
by augmenting the dataset. Both require a priori knowledge about the exact
nature of the symmetries. Absent this knowledge, practitioners resort to
expensive and time-consuming tuning. To address this problem, we propose a new
approach to learn distributions of augmentation transforms, in a new
\emph{Transformed Risk Minimization} (TRM) framework. In addition to predictive
models, we also optimize over transformations chosen from a hypothesis space.
As an algorithmic framework, our TRM method is (1) efficient (jointly learns
augmentations and models in a \emph{single training loop}), (2) modular (works
with \emph{any} training algorithm), and (3) general (handles \emph{both
discrete and continuous} augmentations). We theoretically compare TRM with
standard risk minimization, and give a PAC-Bayes upper bound on its
generalization error. We propose to optimize this bound over a rich
augmentation space via a new parametrization over compositions of blocks,
leading to the new \emph{Stochastic Compositional Augmentation Learning}
(SCALE) algorithm. We compare SCALE experimentally with prior methods (Fast
AutoAugment and Augerino) on CIFAR10/100, SVHN . Additionally, we show that
SCALE can correctly learn certain symmetries in the data distribution
(recovering rotations on rotated MNIST) and can also improve calibration of the
learned model.

提出了一种新的 Transformed Risk Minimization (TRM) 方法，该方法通过学习数据增强变换的分布来解决机器学习中的数据分布结构适应问题；并且通过一种新的参数化组合块的方法，利用 PAC-Bayes upper bound 优化数据增强空格，在 CIFAR10/100、SVHN 等数据集上使用 Stochastic Compositional Augmentation Learning (SCALE) 算法进行实验，提高了学习模型的校准性能。