Most current approaches for protecting privacy in machine learning (ML)
assume that models exist in a vacuum, when in reality, ML models are part of
larger systems that include components for training data filtering, output
monitoring, and more. In this work, we introduce privacy side channels: attacks
that exploit these system-level components to extract private information at
far higher rates than is otherwise possible for standalone models. We propose
four categories of side channels that span the entire ML lifecycle (training
data filtering, input preprocessing, output post-processing, and query
filtering) and allow for either enhanced membership inference attacks or even
novel threats such as extracting users' test queries. For example, we show that
deduplicating training data before applying differentially-private training
creates a side-channel that completely invalidates any provable privacy
guarantees. Moreover, we show that systems which block language models from
regenerating training data can be exploited to allow exact reconstruction of
private keys contained in the training set -- even if the model did not
memorize these keys. Taken together, our results demonstrate the need for a
holistic, end-to-end privacy analysis of machine learning.

通过引入隐私侧信道攻击，本研究揭示了当前保护机器学习隐私的方法假设模型存在于真空中，然而事实上，机器学习模型是包含用于训练数据过滤、输出监测等组件的更大系统的一部分。文中提出了四类隐私侧信道攻击，涵盖了整个机器学习生命周期，可以用于增强成员推断攻击或者进行提取用户测试查询等新颖威胁。通过实例展示，文中指出在应用差分隐私训练之前对训练数据进行去重造成了一种侧信道攻击，完全破坏了任何可证明的隐私保护保证。此外，研究还发现，阻止语言模型再生成训练数据的系统可以被利用来精确重构出包含在训练集中的私钥，即使模型本身并没有记忆这些私钥。总之，本研究的结果表明需要进行全面的端到端的机器学习隐私分析。