Maximum entropy (Maxent) models are a class of statistical models that use
the maximum entropy principle to estimate probability distributions from data.
Due to the size of modern data sets, Maxent models need efficient optimization
algorithms to scale well for big data applications. State-of-the-art algorithms
for Maxent models, however, were not originally designed to handle big data
sets; these algorithms either rely on technical devices that may yield
unreliable numerical results, scale poorly, or require smoothness assumptions
that many practical Maxent models lack. In this paper, we present novel
optimization algorithms that overcome the shortcomings of state-of-the-art
algorithms for training large-scale, non-smooth Maxent models. Our proposed
first-order algorithms leverage the Kullback-Leibler divergence to train
large-scale and non-smooth Maxent models efficiently. For Maxent models with
discrete probability distribution of $n$ elements built from samples, each
containing $m$ features, the stepsize parameters estimation and iterations in
our algorithms scale on the order of $O(mn)$ operations and can be trivially
parallelized. Moreover, the strong $\ell_{1}$ convexity of the
Kullback--Leibler divergence allows for larger stepsize parameters, thereby
speeding up the convergence rate of our algorithms. To illustrate the
efficiency of our novel algorithms, we consider the problem of estimating
probabilities of fire occurrences as a function of ecological features in the
Western US MTBS-Interagency wildfire data set. Our numerical results show that
our algorithms outperform the state of the arts by one order of magnitude and
yield results that agree with physical models of wildfire occurrence and
previous statistical analyses of wildfire drivers.

介绍了一种优化算法，有效地训练了大规模、非光滑的最大熵模型，通过考虑西部美国的野火数据集，结果显示与野火发生的物理模型和以往统计分析结果相符，并且性能超过现有算法一个数量级。