We study the problem of guaranteeing Differential Privacy (DP) in
hyper-parameter tuning, a crucial process in machine learning involving the
selection of the best run from several. Unlike many private algorithms,
including the prevalent DP-SGD, the privacy implications of tuning remain
insufficiently understood. Recent works propose a generic private solution for
the tuning process, yet a fundamental question still persists: is the current
privacy bound for this solution tight?
This paper contributes both positive and negative answers to this question.
Initially, we provide studies affirming the current privacy analysis is indeed
tight in a general sense. However, when we specifically study the
hyper-parameter tuning problem, such tightness no longer holds. This is first
demonstrated by applying privacy audit on the tuning process. Our findings
underscore a substantial gap between the current theoretical privacy bound and
the empirical bound derived even under the strongest audit setup.
The gap found is not a fluke. Our subsequent study provides an improved
privacy result for private hyper-parameter tuning due to its distinct
properties. Our privacy results are also more generalizable compared to prior
analyses that are only easily applicable in specific setups.

在超参数调整问题中，提供了关于当前隐私分析是紧密的一般意义上的证实，然而，当专门研究超参数调整问题时，密切性不再成立。此外，通过隐私审计发现了当前理论上的隐私边界与经验边界之间的实质差距，并提供了改进后的隐私结果以及比以往分析更具普适性的结果。

选择泄露隐私的机制：重新审视私密选择和超参数调整的改进结果

How Does Selection Leak Privacy: Revisiting Private Selection and  Improved Results for Hyper-parameter Tuning

We analytically investigate how over-parameterization of models in randomized
machine learning algorithms impacts the information leakage about their
training data. Specifically, we prove a privacy bound for the KL divergence
between model distributions on worst-case neighboring datasets, and explore its
dependence on the initialization, width, and depth of fully connected neural
networks. We find that this KL privacy bound is largely determined by the
expected squared gradient norm relative to model parameters during training.
Notably, for the special setting of linearized network, our analysis indicates
that the squared gradient norm (and therefore the escalation of privacy loss)
is tied directly to the per-layer variance of the initialization distribution.
By using this analysis, we demonstrate that privacy bound improves with
increasing depth under certain initializations (LeCun and Xavier), while
degrades with increasing depth under other initializations (He and NTK). Our
work reveals a complex interplay between privacy and depth that depends on the
chosen initialization distribution. We further prove excess empirical risk
bounds under a fixed KL privacy budget, and show that the interplay between
privacy utility trade-off and depth is similarly affected by the
initialization.

过度参数化和随机化机器学习算法对其训练数据的信息泄露有何影响，通过解析分析我们得出了模型分布间的 KL 散度的隐私界限，并研究其对全连接神经网络的初始化、宽度和深度的依赖性。我们发现，这个 KL 隐私界限很大程度上由训练过程中的模型参数相对于期望梯度范数决定。我们的分析表明，对于线性化网络的特殊设定，梯度范数（从而隐私损失的升级）直接与初始化分布的每层方差相关联。利用这一分析，我们证明了在特定初始化（LeCun 和 Xavier）下随着深度的增加，隐私界限提高，而在其他初始化（He 和 NTK）下随深度的增加，隐私界限降低。我们的工作揭示了隐私与深度之间的复杂相互作用，这取决于所选的初始化分布。我们进一步证明了在固定 KL 隐私预算下的过度经验风险界限，并展示了隐私效用权衡与深度之间的相互作用同样受到初始化方式的影响。