Unsupervised performance estimation, or evaluating how well models perform on
unlabeled data is a difficult task. Recently, a method was proposed by Garg et
al. [2022] which performs much better than previous methods. Their method
relies on having a score function, satisfying certain properties, to map
probability vectors outputted by the classifier to the reals, but it is an open
problem which score function is best. We explore this problem by first showing
that their method fundamentally relies on the ordering induced by this score
function. Thus, under monotone transformations of score functions, their method
yields the same estimate. Next, we show that in the binary classification
setting, nearly all common score functions - the $L^\infty$ norm; the $L^2$
norm; negative entropy; and the $L^2$, $L^1$, and Jensen-Shannon distances to
the uniform vector - all induce the same ordering over probability vectors.
However, this does not hold for higher dimensional settings. We conduct
numerous experiments on well-known NLP data sets and rigorously explore the
performance of different score functions. We conclude that the $L^\infty$ norm
is the most appropriate.

通常用于评估模型性能的无标签数据评估的方法，通过使用得分函数映射概率向量来提高效果，但是需要选用最佳得分函数，实验得出 L^∞范数是最合适的。

概率向量排序与无监督性能评估

On Orderings of Probability Vectors and Unsupervised Performance  Estimation

Zeroth-order optimization is the process of minimizing an objective $f(x)$,
given oracle access to evaluations at adaptively chosen inputs $x$. In this
paper, we present two simple yet powerful GradientLess Descent (GLD) algorithms
that do not rely on an underlying gradient estimate and are numerically stable.
We analyze our algorithm from a novel geometric perspective and present a novel
analysis that shows convergence within an $\epsilon$-ball of the optimum in
$O(kQ\log(n)\log(R/\epsilon))$ evaluations, for any monotone transform of a
smooth and strongly convex objective with latent dimension $k < n$, where the
input dimension is $n$, $R$ is the diameter of the input space and $Q$ is the
condition number. Our rates are the first of its kind to be both 1)
poly-logarithmically dependent on dimensionality and 2) invariant under
monotone transformations. We further leverage our geometric perspective to show
that our analysis is optimal. Both monotone invariance and its ability to
utilize a low latent dimensionality are key to the empirical success of our
algorithms, as demonstrated on BBOB and MuJoCo benchmarks.

该论文提出了两种简单但强大的无梯度估计的梯度下降（GLD）算法，并从一种新的几何视角进行分析，展示了其具有收敛性的优势，并且能够在保证单调变换不变的情况下，利用低的潜在维数来实现优化，对 BBOB 和 MuJoCo 基准测试产生了良好的实证效果。

无梯度下降：高维零阶优化

Gradientless Descent: High-Dimensional Zeroth-Order Optimization

A theorem of McCann shows that for any two absolutely continuous probability
measures on R^d there exists a monotone transformation sending one probability
measure to the other. A consequence of this theorem, relevant to statistics, is
that density estimation can be recast in terms of transformations. In
particular, one can fix any absolutely continuous probability measure, call it
P, and then reparameterize the whole class of absolutely continuous probability
measures as monotone transformations from P. In this paper we utilize this
reparameterization of densities, as monotone transformations from some P, to
construct semiparametric and nonparametric density estimates. We focus our
attention on classes of transformations, developed in the image processing and
computational anatomy literature, which are smooth, invertible and which have
attractive computational properties. The techniques developed for this class of
transformations allow us to show that a penalized maximum likelihood estimate
(PMLE) of a smooth transformation from P exists and has a finite dimensional
characterization, similar to those results found in the spline literature.
These results are derived utilizing an Euler-Lagrange characterization of the
PMLE which also establishes a surprising connection to a generalization of
Stein's lemma for characterizing the normal distribution.

本文利用 McCann 的一个定理，将密度估计重新表述为一组单调变换，具有一些平滑、可逆、具有吸引力的计算性质的变换，通过最大似然估计（MLE）和惩罚最大似然估计（PMLE）来实现内部转化，从而建立了待估计变换的有限维度表示。