Robust statistics traditionally focuses on outliers, or perturbations in
total variation distance. However, a dataset could be corrupted in many other
ways, such as systematic measurement errors and missing covariates. We
generalize the robust statistics approach to consider perturbations under any
Wasserstein distance, and show that robust estimation is possible whenever a
distribution's population statistics are robust under a certain family of
friendly perturbations. This generalizes a property called resilience
previously employed in the special case of mean estimation with outliers. We
justify the generalized resilience property by showing that it holds under
moment or hypercontractive conditions. Even in the total variation case, these
subsume conditions in the literature for mean estimation, regression, and
covariance estimation; the resulting analysis simplifies and sometimes improves
these known results in both population limit and finite-sample rate. Our robust
estimators are based on minimum distance (MD) functionals (Donoho and Liu,
1988), which project onto a set of distributions under a discrepancy related to
the perturbation. We present two approaches for designing MD estimators with
good finite-sample rates: weakening the discrepancy and expanding the set of
distributions. We also present connections to Gao et al. (2019)'s recent
analysis of generative adversarial networks for robust estimation.

通过在任意 Wasserstein 距离下考虑扰动，实现鲁棒统计推断的推广。在这种情况下，我们首次证明了一个称为广义弹性的性质，并通过 Moment 或超对交条件证明了这种弹性的有效性。最后，我们提供了两种设计具有良好有限样本速率的 MD 估计器的不同方法，包括弱化差异和扩展分布集的方法，回顾了与 GAN 领域最近相关的研究结果。

广义韧性与健壮统计学

Generalized Resilience and Robust Statistics

The growth of data, the need for scalability and the complexity of models
used in modern machine learning calls for distributed implementations. Yet, as
of today, distributed machine learning frameworks have largely ignored the
possibility of arbitrary (i.e., Byzantine) failures. In this paper, we study
the robustness to Byzantine failures at the fundamental level of stochastic
gradient descent (SGD), the heart of most machine learning algorithms. Assuming
a set of $n$ workers, up to $f$ of them being Byzantine, we ask how robust can
SGD be, without limiting the dimension, nor the size of the parameter space.
We first show that no gradient descent update rule based on a linear
combination of the vectors proposed by the workers (i.e, current approaches)
tolerates a single Byzantine failure. We then formulate a resilience property
of the update rule capturing the basic requirements to guarantee convergence
despite $f$ Byzantine workers. We finally propose Krum, an update rule that
satisfies the resilience property aforementioned. For a $d$-dimensional
learning problem, the time complexity of Krum is $O(n^2 \cdot (d + \log n))$.

本文研究了分布式机器学习中的拜占庭错误容错问题，提出了基于 Krum 的更新规则来保证算法的鲁棒性和收敛性。