We study the generalization error of statistical learning algorithms in a
non-i.i.d. setting, where the training data is sampled from a stationary mixing
process. We develop an analytic framework for this scenario based on a
reduction to online learning with delayed feedback. In particular, we show that
the existence of an online learning algorithm with bounded regret (against a
fixed statistical learning algorithm in a specially constructed game of online
learning with delayed feedback) implies low generalization error of said
statistical learning method even if the data sequence is sampled from a mixing
time series. The rates demonstrate a trade-off between the amount of delay in
the online learning game and the degree of dependence between consecutive data
points, with near-optimal rates recovered in a number of well-studied settings
when the delay is tuned appropriately as a function of the mixing time of the
process.

通过将训练数据从一个混合过程进行采样，我们研究了统计学习算法在非独立同分布的环境中的泛化误差，并基于延迟反馈的在线学习提出了一个分析框架。特别地，我们展示了即使数据序列是从混合时间序列采样得到的情况下，通过存在一个具有有界遗憾的在线学习算法（针对一个固定的统计学习算法，在一个特殊构建的延迟反馈的在线学习博弈中），所述统计学习方法的泛化误差较低。这些速率展示了在线学习博弈中的延迟量和连续数据点之间的依赖程度之间的权衡，当延迟适当地调整为过程的混合时间的函数时，可以在许多研究良好的情形下恢复近乎最优的速率。

通过延迟的在线到 PAC 转换推导混合过程的归纳界

Generalization bounds for mixing processes via delayed online-to-PAC  conversions

Generative data augmentation, which scales datasets by obtaining fake labeled
examples from a trained conditional generative model, boosts classification
performance in various learning tasks including (semi-)supervised learning,
few-shot learning, and adversarially robust learning. However, little work has
theoretically investigated the effect of generative data augmentation. To fill
this gap, we establish a general stability bound in this not independently and
identically distributed (non-i.i.d.) setting, where the learned distribution is
dependent on the original train set and generally not the same as the true
distribution. Our theoretical result includes the divergence between the
learned distribution and the true distribution. It shows that generative data
augmentation can enjoy a faster learning rate when the order of divergence term
is $o(\max\left( \log(m)\beta_m, 1 / \sqrt{m})\right)$, where $m$ is the train
set size and $\beta_m$ is the corresponding stability constant. We further
specify the learning setup to the Gaussian mixture model and generative
adversarial nets. We prove that in both cases, though generative data
augmentation does not enjoy a faster learning rate, it can improve the learning
guarantees at a constant level when the train set is small, which is
significant when the awful overfitting occurs. Simulation results on the
Gaussian mixture model and empirical results on generative adversarial nets
support our theoretical conclusions. Our code is available at
this https URL

研究表明，生成数据增强应用于特定学习任务时，如（半）监督学习，少样本学习和敌对鲁棒学习等，可以提高分类性能。本文深入研究了生成数据增强在非独立同分布设置中的表现，证明了其学习速度可以更快，而且在训练集较小的情况下可以提高学习保证级别。