The ability to represent and compare machine learning models is crucial in
order to quantify subtle model changes, evaluate generative models, and gather
insights on neural network architectures. Existing techniques for comparing
data distributions focus on global data properties such as mean and covariance;
in that sense, they are extrinsic and uni-scale. We develop a first-of-its-kind
intrinsic and multi-scale method for characterizing and comparing data
manifolds, using a lower-bound of the spectral variant of the
Gromov-Wasserstein inter-manifold distance, which compares all data moments. In
a thorough experimental study, we demonstrate that our method effectively
discerns the structure of data manifolds even on unaligned data of different
dimensionalities; moreover, we showcase its efficacy in evaluating the quality
of generative models.

用 Gromov-Wasserstein 距离的下界，通过对所有数据矩计算，基于内在和多尺度的方法对比数据流形。实验证明，该方法能够有效地识别不同维度未对齐数据的结构，并展示了在评估生成模型质量方面的功效。

数据形态：数据分布的内在距离

The Shape of Data: Intrinsic Distance for Data Distributions

We propose the first contextual bandit algorithm that is parameter-free,
efficient, and optimal in terms of dynamic regret. Specifically, our algorithm
achieves dynamic regret $\mathcal{O}(\min\{\sqrt{ST},
\Delta^{\frac{1}{3}}T^{\frac{2}{3}}\})$ for a contextual bandit problem with
$T$ rounds, $S$ switches and $\Delta$ total variation in data distributions.
Importantly, our algorithm is adaptive and does not need to know $S$ or
$\Delta$ ahead of time, and can be implemented efficiently assuming access to
an ERM oracle.
Our results strictly improve the $\mathcal{O}(\min
\{S^{\frac{1}{4}}T^{\frac{3}{4}}, \Delta^{\frac{1}{5}}T^{\frac{4}{5}}\})$ bound
of (Luo et al., 2018), and greatly generalize and improve the
$\mathcal{O}(\sqrt{ST})$ result of (Auer et al, 2018) that holds only for the
two-armed bandit problem without contextual information. The key novelty of our
algorithm is to introduce replay phases, in which the algorithm acts according
to its previous decisions for a certain amount of time in order to detect
non-stationarity while maintaining a good balance between exploration and
exploitation.

提出了首个无需参数的、高效的、动态遗憾最优的上下文赌博算法，通过引入回放阶段来保持对非平稳的探索，并在探索和开发之间保持良好的平衡。

一种新的非平稳情境赌博算法：高效、最优和免参数

A New Algorithm for Non-stationary Contextual Bandits: Efficient,  Optimal, and Parameter-free

Generative adversarial networks (GANs) are innovative techniques for learning
generative models of complex data distributions from samples. Despite
remarkable recent improvements in generating realistic images, one of their
major shortcomings is the fact that in practice, they tend to produce samples
with little diversity, even when trained on diverse datasets. This phenomenon,
known as mode collapse, has been the main focus of several recent advances in
GANs. Yet there is little understanding of why mode collapse happens and why
existing approaches are able to mitigate mode collapse. We propose a principled
approach to handling mode collapse, which we call packing. The main idea is to
modify the discriminator to make decisions based on multiple samples from the
same class, either real or artificially generated. We borrow analysis tools
from binary hypothesis testing---in particular the seminal result of Blackwell
[Bla53]---to prove a fundamental connection between packing and mode collapse.
We show that packing naturally penalizes generators with mode collapse, thereby
favoring generator distributions with less mode collapse during the training
process. Numerical experiments on benchmark datasets suggests that packing
provides significant improvements in practice as well.

本文提出了一种名为 packing 的方法来处理 GAN 中的模式崩溃现象，其主要思想是修改判别器使其决策基于同一类别的多个样本（真实或者人工生成的）。数值实验表明 packing 在实践中提供了显著的改进。