Gradient inversion attacks are an ubiquitous threat in federated learning as they exploit gradient leakage to reconstruct supposedly private training data. Recent work has proposed to prevent gradient leakage without loss of model utility by incorporating a PRivacy EnhanCing mODulE (PRECODE) based on variational modeling. Without further analysis, it was shown that PRECODE successfully protects against gradient inversion attacks. In this paper, we make multiple contributions. First, we investigate the effect of PRECODE on gradient inversion attacks to reveal its underlying working principle. We show that variational modeling introduces stochasticity into the gradients of PRECODE and the subsequent layers in a neural network. The stochastic gradients of these layers prevent iterative gradient inversion attacks from converging. Second, we formulate an attack that disables the privacy preserving effect of PRECODE by purposefully omitting stochastic gradients during attack optimization. To preserve the privacy preserving effect of PRECODE, our analysis reveals that variational modeling must be placed early in the network. However, early placement of PRECODE is typically not feasible due to reduced model utility and the exploding number of additional model parameters. Therefore, as a third contribution, we propose a novel privacy module -- the Convolutional Variational Bottleneck (CVB) -- that can be placed early in a neural network without suffering from these drawbacks. We conduct an extensive empirical study on three seminal model architectures and six image classification datasets. We find that all architectures are susceptible to gradient leakage attacks, which can be prevented by our proposed CVB. Compared to PRECODE, we show that our novel privacy module requires fewer trainable parameters, and thus computational and communication costs, to effectively preserve privacy.

梯度反转攻击是联邦学习中普遍存在的威胁，利用梯度泄漏重构原本私密的训练数据。最近的研究提出了一种通过结合基于变分建模的隐私增强模块（PRECODE）来防止梯度泄漏而不损失模型效用的方法。本文的多项贡献包括：揭示了PRECODE对梯度反转攻击的作用原理，提出了一个能在神经网络中早期放置且不受限制的新型隐私模块Convolutional Variational Bottleneck (CVB)，并通过对三个重要模型结构和六个图像分类数据集的广泛实证研究发现，我们的CVB能够有效预防梯度泄漏攻击且参数更少，从而减少计算和通信成本并有效地保护隐私。

具有卷积变分瓶颈的隐私保护联邦学习