The state-of-the-art methods for estimating high-dimensional covariance
matrices all shrink the eigenvalues of the sample covariance matrix towards a
data-insensitive shrinkage target. The underlying shrinkage transformation is
either chosen heuristically - without compelling theoretical justification - or
optimally in view of restrictive distributional assumptions. In this paper, we
propose a principled approach to construct covariance estimators without
imposing restrictive assumptions. That is, we study distributionally robust
covariance estimation problems that minimize the worst-case Frobenius error
with respect to all data distributions close to a nominal distribution, where
the proximity of distributions is measured via a divergence on the space of
covariance matrices. We identify mild conditions on this divergence under which
the resulting minimizers represent shrinkage estimators. We show that the
corresponding shrinkage transformations are intimately related to the
geometrical properties of the underlying divergence. We also prove that our
robust estimators are efficiently computable and asymptotically consistent and
that they enjoy finite-sample performance guarantees. We exemplify our general
methodology by synthesizing explicit estimators induced by the
Kullback-Leibler, Fisher-Rao, and Wasserstein divergences. Numerical
experiments based on synthetic and real data show that our robust estimators
are competitive with state-of-the-art estimators.

提出了一种在不强加限制性假设的情况下构建协方差估计器的原则方法，通过最小化与接近名义分布的所有数据分布相关的最坏情况 Frobenius 误差来研究分布鲁棒协方差估计问题，证明了鲁棒估计器的有效计算性、渐近一致性和有限样本性能保证，并通过合成 Kullback-Leibler、Fisher-Rao 和 Wasserstein 散度的显式估计器来说明这一通用方法。基于合成和实际数据的数值实验表明，我们的鲁棒估计器与最先进的估计器具有相竞争的性能。

通过扩大模糊集合缩小谱值：一种分布鲁棒协方差估计的几何统一

A Geometric Unification of Distributionally Robust Covariance  Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set

Stein's paradox holds considerable sway in high-dimensional statistics,
highlighting that the sample mean, traditionally considered the de facto
estimator, might not be the most efficacious in higher dimensions. To address
this, the James-Stein estimator proposes an enhancement by steering the sample
means toward a more centralized mean vector. In this paper, first, we establish
that normalization layers in deep learning use inadmissible estimators for mean
and variance. Next, we introduce a novel method to employ the James-Stein
estimator to improve the estimation of mean and variance within normalization
layers. We evaluate our method on different computer vision tasks: image
classification, semantic segmentation, and 3D object classification. Through
these evaluations, it is evident that our improved normalization layers
consistently yield superior accuracy across all tasks without extra
computational burden. Moreover, recognizing that a plethora of shrinkage
estimators surpass the traditional estimator in performance, we study two other
prominent shrinkage estimators: Ridge and LASSO. Additionally, we provide
visual representations to intuitively demonstrate the impact of shrinkage on
the estimated layer statistics. Finally, we study the effect of regularization
and batch size on our modified batch normalization. The studies show that our
method is less sensitive to batch size and regularization, improving accuracy
under various setups.

在本文中，我们首先确定深度学习中的归一化层使用的均值和方差估计器不可接受，然后介绍了一种运用 James-Stein 估计器改进均值和方差估计的新方法，评估表明我们改进的归一化层在各种计算机视觉任务中始终具有优越的准确性，而且不增加额外的计算负担。此外，我们还研究了两种显著的缩小估计器：Ridge 和 LASSO，通过可视化表示直观地展示了缩小对估计层统计量的影响，并对正则化和批量大小对我们修改后的批量归一化的影响进行了研究，研究显示我们的方法对批量大小和正则化不太敏感，在不同设置下提高了准确性。

使用 James-Stein 估计器改善归一化

Improving Normalization with the James-Stein Estimator

We develop and analyze empirical Bayes Stein-type estimators for use in the
estimation of causal effects in large-scale online experiments. While online
experiments are generally thought to be distinguished by their large sample
size, we focus on the multiplicity of treatment groups. The typical analysis
practice is to use simple differences-in-means (perhaps with covariate
adjustment) as if all treatment arms were independent. In this work we develop
consistent, small bias, shrinkage estimators for this setting. In addition to
achieving lower mean squared error these estimators retain important
frequentist properties such as coverage under most reasonable scenarios. Modern
sequential methods of experimentation and optimization such as multi-armed
bandit optimization (where treatment allocations adapt over time to prior
responses) benefit from the use of our shrinkage estimators. Exploration under
empirical Bayes focuses more efficiently on near-optimal arms, improving the
resulting decisions made under uncertainty. We demonstrate these properties by
examining seventeen large-scale experiments conducted on Facebook from April to
June 2017.

本研究开发并分析了经验贝叶斯斯坦型估计量，用于在大规模在线实验中估计因果效应。我们通过对 Facebook 在 2017 年 4 月至 6 月进行的 17 个大规模实验的研究来展示这些性质。