In recent years, formal methods of privacy protection such as differential
privacy (DP), capable of deployment to data-driven tasks such as machine
learning (ML), have emerged. Reconciling large-scale ML with the closed-form
reasoning required for the principled analysis of individual privacy loss
requires the introduction of new tools for automatic sensitivity analysis and
for tracking an individual's data and their features through the flow of
computation. For this purpose, we introduce a novel \textit{hybrid} automatic
differentiation (AD) system which combines the efficiency of reverse-mode AD
with an ability to obtain a closed-form expression for any given quantity in
the computational graph. This enables modelling the sensitivity of arbitrary
differentiable function compositions, such as the training of neural networks
on private data. We demonstrate our approach by analysing the individual DP
guarantees of statistical database queries. Moreover, we investigate the
application of our technique to the training of DP neural networks. Our
approach can enable the principled reasoning about privacy loss in the setting
of data processing, and further the development of automatic sensitivity
analysis and privacy budgeting systems.

介绍了一种新的混合自动微分系统，可用于推理隐私损失，并进一步开发自动灵敏度分析和隐私预算系统，在统计数据库查询和 DP 神经网络的训练中具有良好的应用前景。

使用混合自动微分进行差分隐私机器学习的灵敏度分析

Sensitivity analysis in differentially private machine learning using  hybrid automatic differentiation

We consider the problem of reinforcing federated learning with formal privacy
guarantees. We propose to employ Bayesian differential privacy, a relaxation of
differential privacy for similarly distributed data, to provide sharper privacy
loss bounds. We adapt the Bayesian privacy accounting method to the federated
setting and suggest multiple improvements for more efficient privacy budgeting
at different levels. Our experiments show significant advantage over the
state-of-the-art differential privacy bounds for federated learning on image
classification tasks, including a medical application, bringing the privacy
budget below 1 at the client level, and below 0.1 at the instance level. Lower
amounts of noise also benefit the model accuracy and reduce the number of
communication rounds.

本文提出采用贝叶斯差分隐私来增强联邦学习的形式化隐私保证，提出适用于联邦学习的贝叶斯隐私计算方法，并在图像分类任务上表现出明显的优势。