Given a default distribution $P$ and a set of test data
$x^M=\{x_1,x_2,\ldots,x_M\}$ this paper seeks to answer the question if it was
likely that $x^M$ was generated by $P$. For discrete distributions, the
definitive answer is in principle given by Kolmogorov-Martin-L\"{o}f
randomness. In this paper we seek to generalize this to continuous
distributions. We consider a set of statistics $T_1(x^M),T_2(x^M),\ldots$. To
each statistic we associate its maximum entropy distribution and with this a
universal source coder. The maximum entropy distributions are subsequently
combined to give a total codelength, which is compared with $-\log P(x^M)$. We
show that this approach satisfied a number of theoretical properties.
For real world data $P$ usually is unknown. We transform data into a standard
distribution in the latent space using a bidirectional generate network and use
maximum entropy coding there. We compare the resulting method to other methods
that also used generative neural networks to detect anomalies. In most cases,
our results show better performance.

使用默认分布和一组测试数据，本文探讨了是否可能由 P 生成 x^M。为离散分布问题，Kolmogorov-Martin-Löf 随机性提供了明确答案。我们试图将这个概念推广到连续分布，并考虑了一组统计量 T_1 (x^M),T_2 (x^M),...。我们将每个统计量与其最大熵分布以及相应的通用源编码器关联起来，然后将最大熵分布组合以得到总码长，并与 - logP (x^M) 进行比较。我们展示了这种方法满足一些理论性质。对于真实世界的数据，通常 P 是未知的。我们使用双向生成网络将数据转换为潜在空间中的标准分布，并在那里使用最大熵编码。我们将所得方法与其他使用生成型神经网络检测异常值的方法进行了比较，在大多数情况下，我们的结果表现更好。

最大熵编码的异常检测

Out-of-Distribution Detection using Maximum Entropy Coding

Scientific modeling applications often require estimating a distribution of
parameters consistent with a dataset of observations - an inference task also
known as source distribution estimation. This problem can be ill-posed,
however, since many different source distributions might produce the same
distribution of data-consistent simulations. To make a principled choice among
many equally valid sources, we propose an approach which targets the maximum
entropy distribution, i.e., prioritizes retaining as much uncertainty as
possible. Our method is purely sample-based - leveraging the Sliced-Wasserstein
distance to measure the discrepancy between the dataset and simulations - and
thus suitable for simulators with intractable likelihoods. We benchmark our
method on several tasks, and show that it can recover source distributions with
substantially higher entropy without sacrificing the fidelity of the
simulations. Finally, to demonstrate the utility of our approach, we infer
source distributions for parameters of the Hodgkin-Huxley neuron model from
experimental datasets with thousands of measurements. In summary, we propose a
principled framework for inferring unique source distributions of scientific
simulator parameters while retaining as much uncertainty as possible.

我们提出了一种用于推断科学模拟器参数的唯一源分布的方法，并尽可能保留更多的不确定性。

魔术师：基于样本的最大熵源分布估计

Sourcerer: Sample-based Maximum Entropy Source Distribution Estimation

We show that the herding procedure of Welling (2009) takes exactly the form
of a standard convex optimization algorithm--namely a conditional gradient
algorithm minimizing a quadratic moment discrepancy. This link enables us to
invoke convergence results from convex optimization and to consider faster
alternatives for the task of approximating integrals in a reproducing kernel
Hilbert space. We study the behavior of the different variants through
numerical simulations. The experiments indicate that while we can improve over
herding on the task of approximating integrals, the original herding algorithm
tends to approach more often the maximum entropy distribution, shedding more
light on the learning bias behind herding.

本文研究了 Welling（2009）的聚集过程，发现其采用标准的凸优化算法，即条件渐近算法，通过最小化二次矩差异。通过数值模拟研究了不同变体的行为。结果表明，虽然我们可以在近似积分的任务上对 herding 进行改进，但原始的 herding 算法更倾向于接近最大熵分布，揭示了 herding 背后的学习偏差。