Recent studies have shown that distributed machine learning is vulnerable to
gradient inversion attacks, where private training data can be reconstructed by
analyzing the gradients of the models shared in training. Previous attacks
established that such reconstructions are possible using gradients from all
parameters in the entire models. However, we hypothesize that most of the
involved modules, or even their sub-modules, are at risk of training data
leakage, and we validate such vulnerabilities in various intermediate layers of
language models. Our extensive experiments reveal that gradients from a single
Transformer layer, or even a single linear component with 0.54% parameters, are
susceptible to training data leakage. Additionally, we show that applying
differential privacy on gradients during training offers limited protection
against the novel vulnerability of data disclosure.

分布式机器学习中的梯度反演攻击问题及其对隐私数据的泄露风险进行研究和验证，并发现在语言模型的各个中间层存在训练数据泄露的漏洞。实验证实，单个 Transformer 层甚至一个线性组件的 0.54% 参数的梯度也存在训练数据泄露的问题。此外，研究还表明，在训练过程中对梯度应用差分隐私仅能提供有限的数据泄露保护。

透过树木看到森林：部分变压器梯度的数据泄漏

Seeing the Forest through the Trees: Data Leakage from Partial  Transformer Gradients

Model Inversion (MI) attacks aim to disclose private information about the
training data by abusing access to the pre-trained models. These attacks enable
adversaries to reconstruct high-fidelity data that closely aligns with the
private training data, which has raised significant privacy concerns. Despite
the rapid advances in the field, we lack a comprehensive overview of existing
MI attacks and defenses. To fill this gap, this paper thoroughly investigates
this field and presents a holistic survey. Firstly, our work briefly reviews
the traditional MI on machine learning scenarios. We then elaborately analyze
and compare numerous recent attacks and defenses on \textbf{D}eep
\textbf{N}eural \textbf{N}etworks (DNNs) across multiple modalities and
learning tasks.

模型反演攻击致力于利用对预训练模型的访问权限揭示关于训练数据的私密信息，这些攻击使得对与私密训练数据密切一致的高保真数据的重建成为可能，从而引发了重大的隐私担忧。尽管该领域取得了快速的进展，但我们仍然缺乏对现有模型反演攻击和防御方法的全面概述。为了填补这一空白，本文对该领域进行了深入研究，并提出了综合调查报告。首先，本文简要回顾了机器学习场景下传统的模型反演方法。然后，对多种模态和学习任务下深度神经网络 (DNNs) 的近期攻击和防御方法进行了详细分析和比较。

DNN 隐私泄露：模型倒置攻击和防御方法的调查

Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and  Defenses

It is widely believed that sharing gradients will not leak private training
data in distributed learning systems such as Collaborative Learning and
Federated Learning, etc. Recently, Zhu et al. presented an approach which shows
the possibility to obtain private training data from the publicly shared
gradients. In their Deep Leakage from Gradient (DLG) method, they synthesize
the dummy data and corresponding labels with the supervision of shared
gradients. However, DLG has difficulty in convergence and discovering the
ground-truth labels consistently. In this paper, we find that sharing gradients
definitely leaks the ground-truth labels. We propose a simple but reliable
approach to extract accurate data from the gradients. Particularly, our
approach can certainly extract the ground-truth labels as opposed to DLG, hence
we name it Improved DLG (iDLG). Our approach is valid for any differentiable
model trained with cross-entropy loss over one-hot labels. We mathematically
illustrate how our method can extract ground-truth labels from the gradients
and empirically demonstrate the advantages over DLG.

本文提出了一种名为 iDLG 的改良方法，该方法能够从共享梯度中提取与 DLG 方法不同的正确标签，并且该方法适用于使用交叉熵损失训练的任何可微分模型，具有优越性能。