Standard federated optimization methods successfully apply to stochastic
problems with single-level structure. However, many contemporary ML problems --
including adversarial robustness, hyperparameter tuning, and actor-critic --
fall under nested bilevel programming that subsumes minimax and compositional
optimization. In this work, we propose \fedblo: A federated alternating
stochastic gradient method to address general nested problems. We establish
provable convergence rates for \fedblo in the presence of heterogeneous data
and introduce variations for bilevel, minimax, and compositional optimization.
\fedblo introduces multiple innovations including federated hypergradient
computation and variance reduction to address inner-level heterogeneity. We
complement our theory with experiments on hyperparameter \&
hyper-representation learning and minimax optimization that demonstrate the
benefits of our method in practice. Code is available at
this https URL

本文提出了一种名为 edblo 的联邦交替随机梯度方法，用于解决复杂的嵌套双层规划问题，并在异构数据情况下建立了收敛性分析，同时提出了多种升级方法，包括联邦超梯度计算和方差缩减，实验结果表明其在超参和超表示学习和最小化最大化规划中具有实际效益。