We introduce MESSY estimation, a Maximum-Entropy based Stochastic and
Symbolic densitY estimation method. The proposed approach recovers probability
density functions symbolically from samples using moments of a Gradient flow in
which the ansatz serves as the driving force. In particular, we construct a
gradient-based drift-diffusion process that connects samples of the unknown
distribution function to a guess symbolic expression. We then show that when
the guess distribution has the maximum entropy form, the parameters of this
distribution can be found efficiently by solving a linear system of equations
constructed using the moments of the provided samples. Furthermore, we use
Symbolic regression to explore the space of smooth functions and find optimal
basis functions for the exponent of the maximum entropy functional leading to
good conditioning. The cost of the proposed method in each iteration of the
random search is linear with the number of samples and quadratic with the
number of basis functions. We validate the proposed MESSY estimation method
against other benchmark methods for the case of a bi-modal and a discontinuous
density, as well as a density at the limit of physical realizability. We find
that the addition of a symbolic search for basis functions improves the
accuracy of the estimation at a reasonable additional computational cost. Our
results suggest that the proposed method outperforms existing density recovery
methods in the limit of a small to moderate number of samples by providing a
low-bias and tractable symbolic description of the unknown density at a
reasonable computational cost.

本文提出了基于最大熵的随机和符号密度估计方法，该方法通过符号梯度流从样本中恢复概率密度函数，进而通过构建由样本猜测符号表达式的梯度漂移扩散过程并解决用样本的矩所构建的线性方程组，找到猜测分布为最大熵形式时分布的参数，使用符号回归找到最优基函数以提高极大熵函数的指数的条件数，该方法在计算代价方面具有优势。