We consider estimating the Shannon entropy of a discrete distribution $P$
from $n$ i.i.d. samples. Recently, Jiao, Venkat, Han, and Weissman, and Wu and
Yang constructed approximation theoretic estimators that achieve the minimax
$L_2$ rates in estimating entropy. Their estimators are consistent given $n \gg
\frac{S}{\ln S}$ samples, where $S$ is the alphabet size, and it is the best
possible sample complexity. In contrast, the Maximum Likelihood Estimator
(MLE), which is the empirical entropy, requires $n\gg S$ samples.
In the present paper we significantly refine the minimax results of existing
work. To alleviate the pessimism of minimaxity, we adopt the adaptive
estimation framework, and show that the minimax rate-optimal estimator in Jiao,
Venkat, Han, and Weissman achieves the minimax rates simultaneously over a
nested sequence of subsets of distributions $P$, without knowing the alphabet
size $S$ or which subset $P$ lies in. In other words, their estimator is
adaptive with respect to this nested sequence of the parameter space, which is
characterized by the entropy of the distribution. We also characterize the
maximum risk of the MLE over this nested sequence, and show, for every subset
in the sequence, that the performance of the minimax rate-optimal estimator
with $n$ samples is essentially that of the MLE with $n\ln n$ samples, thereby
further substantiating the generality of the phenomenon identified by Jiao,
Venkat, Han, and Weissman.

本研究针对离散分布 P 进行 n 个独立同分布样本的香农熵估计，使用逼近理论法进行估计，实现了在估计熵的最小二乘率方面的极致。通过采用自适应估计框架，该方法相对极小值优化估计方法在分布 P 的嵌套子序列上实现了最小二乘率的估计，从而进一步证明了估计在样本 n 的情况下是最优的，并且基本上相当于 MLE 使用 nlnn 个样本进行估计。