The problem of adversarial examples has highlighted the need for a theory of
regularisation that is general enough to apply to exotic function classes, such
as universal approximators. In response, we give a very general equality result
regarding the relationship between distributional robustness and
regularisation, as defined with a transportation cost uncertainty set. The
theory allows us to (tightly) certify the robustness properties of a
Lipschitz-regularised model with very mild assumptions. As a theoretical
application we show a new result explicating the connection between adversarial
learning and distributional robustness. We then give new results for how to
achieve Lipschitz regularisation of kernel classifiers, which are demonstrated
experimentally.

探讨关于对抗性样本的问题，提出一种基于运输成本不确定性集合的正则化方法，实现了对 Lipschitz 正则化模型的稳健性证明，揭示了对抗学习和分布鲁棒性之间的联系，并提供了对核分类器的 Lipschitz 正则化方法。

广义 Lipschitz 正则化等于分布式健壮性

Generalised Lipschitz Regularisation Equals Distributional Robustness

Data augmentation, a technique in which a training set is expanded with
class-preserving transformations, is ubiquitous in modern machine learning
pipelines. In this paper, we seek to establish a theoretical framework for
understanding data augmentation. We approach this from two directions: First,
we provide a general model of augmentation as a Markov process, and show that
kernels appear naturally with respect to this model, even when we do not employ
kernel classification. Next, we analyze more directly the effect of
augmentation on kernel classifiers, showing that data augmentation can be
approximated by first-order feature averaging and second-order variance
regularization components. These frameworks both serve to illustrate the ways
in which data augmentation affects the downstream learning model, and the
resulting analyses provide novel connections between prior work in invariant
kernels, tangent propagation, and robust optimization. Finally, we provide
several proof-of-concept applications showing that our theory can be useful for
accelerating machine learning workflows, such as reducing the amount of
computation needed to train using augmented data, and predicting the utility of
a transformation prior to training.

本文提出了一个理论框架来理解数据增强技术，并从马尔科夫过程和核分类器两个方向进行分析。研究发现，数据增强可以通过一阶特征平均和二阶方差正则化组件来实现近似。本文还将理论应用于加速机器学习工作流，并证明其在预测变换效用和减少使用增强数据所需计算量方面具有实用价值。

现代数据增强的内核理论

A Kernel Theory of Modern Data Augmentation

Kernel classifiers and regressors designed for structured data, such as
sequences, trees and graphs, have significantly advanced a number of
interdisciplinary areas such as computational biology and drug design.
Typically, kernels are designed beforehand for a data type which either exploit
statistics of the structures or make use of probabilistic generative models,
and then a discriminative classifier is learned based on the kernels via convex
optimization. However, such an elegant two-stage approach also limited kernel
methods from scaling up to millions of data points, and exploiting
discriminative information to learn feature representations.
We propose, structure2vec, an effective and scalable approach for structured
data representation based on the idea of embedding latent variable models into
feature spaces, and learning such feature spaces using discriminative
information. Interestingly, structure2vec extracts features by performing a
sequence of function mappings in a way similar to graphical model inference
procedures, such as mean field and belief propagation. In applications
involving millions of data points, we showed that structure2vec runs 2 times
faster, produces models which are $10,000$ times smaller, while at the same
time achieving the state-of-the-art predictive performance.

提出了一种名为 structure2vec 的基于嵌入潜在变量模型的结构化数据表征方法，利用判别性信息进行特征空间学习，在数百万数据点的应用中运行速度是传统方法的两倍，模型则小 $10,000$ 倍，同时达到了最先进的预测性能。