Machine learning models can be trained with formal privacy guarantees via
differentially private optimizers such as DP-SGD. In this work, we study such
privacy guarantees when the adversary only accesses the final model, i.e.,
intermediate model updates are not released. In the existing literature, this
hidden state threat model exhibits a significant gap between the lower bound
provided by empirical privacy auditing and the theoretical upper bound provided
by privacy accounting. To challenge this gap, we propose to audit this threat
model with adversaries that craft a gradient sequence to maximize the privacy
loss of the final model without accessing intermediate models. We demonstrate
experimentally how this approach consistently outperforms prior attempts at
auditing the hidden state model. When the crafted gradient is inserted at every
optimization step, our results imply that releasing only the final model does
not amplify privacy, providing a novel negative result. On the other hand, when
the crafted gradient is not inserted at every step, we show strong evidence
that a privacy amplification phenomenon emerges in the general non-convex
setting (albeit weaker than in convex regimes), suggesting that existing
privacy upper bounds can be improved.

通过对隐含状态威胁模型进行审计，本研究验证了通过在每个优化步骤中插入所精心设计的梯度，保证仅发布最终模型并不会增加隐私的结论，同时也在非凸设置中观察到隐私放大的现象，为改进现有隐私上界提供强有力的证据。