Federated Learning (FL) is a distributed machine learning approach that
enables model training in communication efficient and privacy-preserving
manner. The standard optimization method in FL is Federated Averaging (FedAvg),
which performs multiple local SGD steps between communication rounds. FedAvg
has been considered to lack algorithm adaptivity compared to modern first-order
adaptive optimizations. In this paper, we propose new communication-efficient
FL algortithms based on two adaptive frameworks: local adaptivity (PreFed) and
server-side adaptivity (PreFedOp). Proposed methods adopt adaptivity by using a
novel covariance matrix preconditioner. Theoretically, we provide convergence
guarantees for our algorithms. The empirical experiments show our methods
achieve state-of-the-art performances on both i.i.d. and non-i.i.d. settings.

在本文中，我们提出了基于局部适应性和服务器端适应性的新的通信高效的联邦学习算法，通过使用新型协方差矩阵预处理器，我们的方法在理论上提供了收敛保证，并在 i.i.d. 和非 i.i.d. 设置下取得了最先进的性能。

预处理联邦学习

Preconditioned Federated Learning

Federated Learning (FL) is a machine learning paradigm where local nodes
collaboratively train a central model while the training data remains
decentralized. Existing FL methods typically share model parameters or employ
co-distillation to address the issue of unbalanced data distribution. However,
they suffer from communication bottlenecks. More importantly, they risk privacy
leakage. In this work, we develop a privacy preserving and communication
efficient method in a FL framework with one-shot offline knowledge distillation
using unlabeled, cross-domain public data. We propose a quantized and noisy
ensemble of local predictions from completely trained local models for stronger
privacy guarantees without sacrificing accuracy. Based on extensive experiments
on image classification and text classification tasks, we show that our
privacy-preserving method outperforms baseline FL algorithms with superior
performance in both accuracy and communication efficiency.

本文提出了一种基于联邦学习框架的隐私保护和通信高效方法，使用未标记的跨域公共数据进行一次离线知识蒸馏，提出了一种量化和嘈杂的本地预测集成方法，同时保证了强隐私保证和高准确性。

用集合跨领域知识蒸馏保护联邦学习中的隐私

Preserving Privacy in Federated Learning with Ensemble Cross-Domain  Knowledge Distillation

We study the classic $k$-means/median clustering, which are fundamental
problems in unsupervised learning, in the setting where data are partitioned
across multiple sites, and where we are allowed to discard a small portion of
the data by labeling them as outliers. We propose a simple approach based on
constructing small summary for the original dataset. The proposed method is
time and communication efficient, has good approximation guarantees, and can
identify the global outliers effectively. To the best of our knowledge, this is
the first practical algorithm with theoretical guarantees for distributed
clustering with outliers. Our experiments on both real and synthetic data have
demonstrated the clear superiority of our algorithm against all the baseline
algorithms in almost all metrics.

本文提出了一种基于简要构建的数据摘要的分布式无监督学习算法，对异常点的全局识别有很好的近似保证，并在真实和人工数据上超过了所有基线算法。