Recent studies have shown that distributed machine learning is vulnerable to
gradient inversion attacks, where private training data can be reconstructed by
analyzing the gradients of the models shared in training. Previous attacks
established that such reconstructions are possible using gradients from all
parameters in the entire models. However, we hypothesize that most of the
involved modules, or even their sub-modules, are at risk of training data
leakage, and we validate such vulnerabilities in various intermediate layers of
language models. Our extensive experiments reveal that gradients from a single
Transformer layer, or even a single linear component with 0.54% parameters, are
susceptible to training data leakage. Additionally, we show that applying
differential privacy on gradients during training offers limited protection
against the novel vulnerability of data disclosure.

分布式机器学习中的梯度反演攻击问题及其对隐私数据的泄露风险进行研究和验证，并发现在语言模型的各个中间层存在训练数据泄露的漏洞。实验证实，单个 Transformer 层甚至一个线性组件的 0.54% 参数的梯度也存在训练数据泄露的问题。此外，研究还表明，在训练过程中对梯度应用差分隐私仅能提供有限的数据泄露保护。

透过树木看到森林：部分变压器梯度的数据泄漏

Seeing the Forest through the Trees: Data Leakage from Partial  Transformer Gradients

Federated learning of deep learning models for supervised tasks, e.g. image
classification and segmentation, has found many applications: for example in
human-in-the-loop tasks such as film post-production where it enables sharing
of domain expertise of human artists in an efficient and effective fashion. In
many such applications, we need to protect the training data from being leaked
when gradients are shared in the training process due to IP or privacy
concerns. Recent works have demonstrated that it is possible to reconstruct the
training data from gradients for an image-classification model when its
architecture is known. However, there is still an incomplete theoretical
understanding of the efficacy and failure of such attacks. In this paper, we
analyse the source of training-data leakage from gradients. We formulate the
problem of training data reconstruction as solving an optimisation problem
iteratively for each layer. The layer-wise objective function is primarily
defined by weights and gradients from the current layer as well as the output
from the reconstruction of the subsequent layer, but it might also involve a
'pull-back' constraint from the preceding layer. Training data can be
reconstructed when we solve the problem backward from the output of the network
through each layer. Based on this formulation, we are able to attribute the
potential leakage of the training data in a deep network to its architecture.
We also propose a metric to measure the level of security of a deep learning
model against gradient-based attacks on the training data.

本文分析了基于梯度下降的攻击如何泄露深度学习模型的训练数据，并提出了一种度量安全性的指标，同时也提出了一种用于解决重构训练数据的优化问题的方法。