Detecting overfitting in generative models is an important challenge in
machine learning. In this work, we formalize a form of overfitting that we call
{\em{data-copying}} -- where the generative model memorizes and outputs
training samples or small variations thereof. We provide a three sample
non-parametric test for detecting data-copying that uses the training set, a
separate sample from the target distribution, and a generated sample from the
model, and study the performance of our test on several canonical models and
datasets.
For code \& examples, visit this https URL

研究了在生成模型中检测过拟合的重要挑战，提出了一种形式化的过拟合形式 —— 数据复制，通过一个三个样本的非参数检验来检测数据复制，包含训练集、源分布样本、模型生成样本，并在几个典型模型和数据集上进行了研究。

一种非参数检验方法，用于发现生成模型中的数据复制

A Non-Parametric Test to Detect Data-Copying in Generative Models

Designing codes that combat the noise in a communication medium has remained
a significant area of research in information theory as well as wireless
communications. Asymptotically optimal channel codes have been developed by
mathematicians for communicating under canonical models after over 60 years of
research. On the other hand, in many non-canonical channel settings, optimal
codes do not exist and the codes designed for canonical models are adapted via
heuristics to these channels and are thus not guaranteed to be optimal. In this
work, we make significant progress on this problem by designing a fully
end-to-end jointly trained neural encoder and decoder, namely, Turbo
Autoencoder (TurboAE), with the following contributions: ($a$) under moderate
block lengths, TurboAE approaches state-of-the-art performance under canonical
channels; ($b$) moreover, TurboAE outperforms the state-of-the-art codes under
non-canonical settings in terms of reliability. TurboAE shows that the
development of channel coding design can be automated via deep learning, with
near-optimal performance.

本文介绍了一种新型神经编码器和解码器 TurboAE，该编码器可以通过深度学习来自动化设计通道编码，以提高非规范通道模型的可靠性，并在中等块长度下接近规范通道模型的最新性能水平。

Turbo Autoencoder：基于深度学习的点对点通信信道编码

Turbo Autoencoder: Deep learning based channel codes for point-to-point  communication channels

The mean field methods, which entail approximating intractable probability
distributions variationally with distributions from a tractable family, enjoy
high efficiency, guaranteed convergence, and provide lower bounds on the true
likelihood. But due to requirement for model-specific derivation of the
optimization equations and unclear inference quality in various models, it is
not widely used as a generic approximate inference algorithm. In this paper, we
discuss a generalized mean field theory on variational approximation to a broad
class of intractable distributions using a rich set of tractable distributions
via constrained optimization over distribution spaces. We present a class of
generalized mean field (GMF) algorithms for approximate inference in complex
exponential family models, which entails limiting the optimization over the
class of cluster-factorizable distributions. GMF is a generic method requiring
no model-specific derivations. It factors a complex model into a set of
disjoint variable clusters, and uses a set of canonical fix-point equations to
iteratively update the cluster distributions, and converge to locally optimal
cluster marginals that preserve the original dependency structure within each
cluster, hence, fully decomposed the overall inference problem. We empirically
analyzed the effect of different tractable family (clusters of different
granularity) on inference quality, and compared GMF with BP on several
canonical models. Possible extension to higher-order MF approximation is also
discussed.

在这篇论文中，我们讨论了一种广义均场理论，它使用可追踪分布的丰富集合来近似表示一类难以处理的分布，并通过分布空间上的约束优化来实现。我们还介绍了一类广义均场算法，用于复杂的指数族模型的近似推断，它将复杂模型分解成一组不相交变量簇，并使用一组规范的定点方程来迭代地更新参数，找到保留每个簇内原始依赖结构的局部最优参数，因此完全分解了整体推断问题。通过实验证明，我们比较了不同可追踪族群（不同粒度的簇）对推理质量的影响，并将其与 BP 算法在几个标准模型上进行了比较。还讨论了高阶 MF 逼近的可能扩展。