Decentralized learning is appealing as it enables the scalable usage of large amounts of distributed data and resources (without resorting to any central entity), while promoting privacy since every user minimizes the direct exposure of their data. Yet, without additional precautions, curious users can still leverage models obtained from their peers to violate privacy. In this paper, we propose Decor, a variant of decentralized SGD with differential privacy (DP) guarantees. Essentially, in Decor, users securely exchange randomness seeds in one communication round to generate pairwise-canceling correlated Gaussian noises, which are injected to protect local models at every communication round. We theoretically and empirically show that, for arbitrary connected graphs, Decor matches the central DP optimal privacy-utility trade-off. We do so under SecLDP, our new relaxation of local DP, which protects all user communications against an external eavesdropper and curious users, assuming that every pair of connected users shares a secret, i.e., an information hidden to all others. The main theoretical challenge is to control the accumulation of non-canceling correlated noise due to network sparsity. We also propose a companion SecLDP privacy accountant for public use.

本文提出了Decor，它是一种具有差分隐私保证的分散式 SGD 变体，通过一轮通信中安全地交换随机种子来生成成对抵消的相关高斯噪声，以保护每一轮通信中的本地模型，并在理论上和实践上证明Decor在任意连通图中匹配了中心化差分隐私的最佳隐私-效用折衷，并在SecLDP下实现了所有用户通信的保护，假设每对连接的用户共享一个密钥作为隐藏信息。

去中心化学习中相关噪音的隐私保护能力