Federated learning (FL) has attracted vivid attention as a privacy-preserving
distributed learning framework. In this work, we focus on cross-silo FL, where
clients become the model owners after training and are only concerned about the
model's generalization performance on their local data. Due to the data
heterogeneity issue, asking all the clients to join a single FL training
process may result in model performance degradation. To investigate the
effectiveness of collaboration, we first derive a generalization bound for each
client when collaborating with others or when training independently. We show
that the generalization performance of a client can be improved only by
collaborating with other clients that have more training data and similar data
distribution. Our analysis allows us to formulate a client utility maximization
problem by partitioning clients into multiple collaborating groups. A
hierarchical clustering-based collaborative training (HCCT) scheme is then
proposed, which does not need to fix in advance the number of groups. We
further analyze the convergence of HCCT for general non-convex loss functions
which unveils the effect of data similarity among clients. Extensive
simulations show that HCCT achieves better generalization performance than
baseline schemes, whereas it degenerates to independent training and
conventional FL in specific scenarios.

分布式联邦学习中，研究探讨了通过分组合作来提高模型的泛化性能，解决了数据异构性问题。

协作方法：为了最大化跨边界联邦学习的泛化性能

How to Collaborate: Towards Maximizing the Generalization Performance in  Cross-Silo Federated Learning

Differentially Private Federated Learning (DP-FL) has garnered attention as a
collaborative machine learning approach that ensures formal privacy. Most DP-FL
approaches ensure DP at the record-level within each silo for cross-silo FL.
However, a single user's data may extend across multiple silos, and the desired
user-level DP guarantee for such a setting remains unknown. In this study, we
present ULDP-FL, a novel FL framework designed to guarantee user-level DP in
cross-silo FL where a single user's data may belong to multiple silos. Our
proposed algorithm directly ensures user-level DP through per-user weighted
clipping, departing from group-privacy approaches. We provide a theoretical
analysis of the algorithm's privacy and utility. Additionally, we enhance the
algorithm's utility and showcase its private implementation using cryptographic
building blocks. Empirical experiments on real-world datasets show substantial
improvements in our methods in privacy-utility trade-offs under user-level DP
compared to baseline methods. To the best of our knowledge, our work is the
first FL framework that effectively provides user-level DP in the general
cross-silo FL setting.

我们提出了一种新的联邦学习框架 ULDP-FL，它通过每个用户的加权剪裁直接保证用户级差分隐私，在通用的跨域联邦学习中有效地提供了用户级差分隐私，并在隐私 - 效用权衡方面相比基线方法有了显著的改进。

跨领域用户级差分隐私的联邦学习 ULDP-FL

ULDP-FL: Federated Learning with Across Silo User-Level Differential  Privacy

Federated learning (FL) is a prospective distributed machine learning
framework that can preserve data privacy. In particular, cross-silo FL can
complete model training by making isolated data islands of different
organizations collaborate with a parameter server (PS) via exchanging model
parameters for multiple communication rounds. In cross-silo FL, an incentive
mechanism is indispensable for motivating data owners to contribute their
models to FL training. However, how to allocate the reward budget among
different rounds is an essential but complicated problem largely overlooked by
existing works. The challenge of this problem lies in the opaque feedback
between reward budget allocation and model utility improvement of FL, making
the optimal reward budget allocation complicated. To address this problem, we
design an online reward budget allocation algorithm using Bayesian optimization
named BARA (\underline{B}udget \underline{A}llocation for \underline{R}everse
\underline{A}uction). Specifically, BARA can model the complicated relationship
between reward budget allocation and final model accuracy in FL based on
historical training records so that the reward budget allocated to each
communication round is dynamically optimized so as to maximize the final model
utility. We further incorporate the BARA algorithm into reverse auction-based
incentive mechanisms to illustrate its effectiveness. Extensive experiments are
conducted on real datasets to demonstrate that BARA significantly outperforms
competitive baselines by improving model utility with the same amount of reward
budget.

提出了一个使用贝叶斯优化算法进行在线奖励预算分配的拍卖激励机制 BARA，用于交叉组织边缘设备的联邦学习优化，其有效性已在真实数据集上得到验证。

BARA: 跨边缘联邦学习中基于在线奖励预算分配的高效激励机制

BARA: Efficient Incentive Mechanism with Online Reward Budget Allocation  in Cross-Silo Federated Learning

Federated learning (FL) is an emerging technology that enables the training
of machine learning models from multiple clients while keeping the data
distributed and private. Based on the participating clients and the model
training scale, federated learning can be classified into two types:
cross-device FL where clients are typically mobile devices and the client
number can reach up to a scale of millions; cross-silo FL where clients are
organizations or companies and the client number is usually small (e.g., within
a hundred). While existing studies mainly focus on cross-device FL, this paper
aims to provide an overview of the cross-silo FL. More specifically, we first
discuss applications of cross-silo FL and outline its major challenges. We then
provide a systematic overview of the existing approaches to the challenges in
cross-silo FL by focusing on their connections and differences to cross-device
FL. Finally, we discuss future directions and open issues that merit research
efforts from the community.

本文概述了联邦学习领域中的一个新兴分支 “cross-silo FL”，分析了其应用和所面临的主要挑战，并对现有方法进行了系统总结，同时指出了未来需要重点研究的方向和问题。