In this work, we consider the distributed stochastic optimization problem of
minimizing a non-convex function $f(x) = \mathbb{E}_{\xi \sim \mathcal{D}} f(x;
\xi)$ in an adversarial setting, where the individual functions $f(x; \xi)$ can
also be potentially non-convex. We assume that at most $\alpha$-fraction of a
total of $K$ nodes can be Byzantines. We propose a robust stochastic
variance-reduced gradient (SVRG) like algorithm for the problem, where the
batch gradients are computed at the worker nodes (WNs) and the stochastic
gradients are computed at the server node (SN). For the non-convex optimization
problem, we show that we need $\tilde{O}\left( \frac{1}{\epsilon^{5/3} K^{2/3}}
+ \frac{\alpha^{4/3}}{\epsilon^{5/3}} \right)$ gradient computations on average
at each node (SN and WNs) to reach an $\epsilon$-stationary point. The proposed
algorithm guarantees convergence via the design of a novel Byzantine filtering
rule which is independent of the problem dimension. Importantly, we capture the
effect of the fraction of Byzantine nodes $\alpha$ present in the network on
the convergence performance of the algorithm.

本研究考虑了在敌对环境下最小化一个期望函数的非凸分布式随机优化问题，在此基础上，提出了一种机制，通过使用一种新型的拜占庭过滤规则，可以保证算法收敛，并且能够捕捉网络中破损节点所带来的影响。