Detecting overfitting in generative models is an important challenge in
machine learning. In this work, we formalize a form of overfitting that we call
{\em{data-copying}} -- where the generative model memorizes and outputs
training samples or small variations thereof. We provide a three sample
non-parametric test for detecting data-copying that uses the training set, a
separate sample from the target distribution, and a generated sample from the
model, and study the performance of our test on several canonical models and
datasets.
For code \& examples, visit this https URL

研究了在生成模型中检测过拟合的重要挑战，提出了一种形式化的过拟合形式 —— 数据复制，通过一个三个样本的非参数检验来检测数据复制，包含训练集、源分布样本、模型生成样本，并在几个典型模型和数据集上进行了研究。

一种非参数检验方法，用于发现生成模型中的数据复制

A Non-Parametric Test to Detect Data-Copying in Generative Models

Conditional independence testing is a fundamental problem underlying causal
discovery and a particularly challenging task in the presence of nonlinear and
high-dimensional dependencies. Here a fully non-parametric test for continuous
data based on conditional mutual information combined with a local permutation
scheme is presented. Through a nearest neighbor approach, the test efficiently
adapts also to non-smooth distributions due to strongly nonlinear dependencies.
Numerical experiments demonstrate that the test reliably simulates the null
distribution even for small sample sizes and with high-dimensional conditioning
sets. The test is better calibrated than kernel-based tests utilizing an
analytical approximation of the null distribution, especially for non-smooth
densities, and reaches the same or higher power levels. Combining the local
permutation scheme with the kernel tests leads to better calibration, but
suffers in power. For smaller sample sizes and lower dimensions, the test is
faster than random fourier feature-based kernel tests if the permutation scheme
is (embarrassingly) parallelized, but the runtime increases more sharply with
sample size and dimensionality. Thus, more theoretical research to analytically
approximate the null distribution and speed up the estimation for larger sample
sizes is desirable.

提出了一种基于条件互信息和局部置换方案的全非参数连续数据测试方法，能够适应强非线性相关性，表现优于基于核的测试，并在小样本和高维条件下可靠地模拟零分布。