It has become common to publish large (billion parameter) language models
that have been trained on private datasets. This paper demonstrates that in
such settings, an adversary can perform a training data extraction attack to
recover individual training examples by querying the language model.
We demonstrate our attack on GPT-2, a language model trained on scrapes of
the public Internet, and are able to extract hundreds of verbatim text
sequences from the model's training data. These extracted examples include
(public) personally identifiable information (names, phone numbers, and email
addresses), IRC conversations, code, and 128-bit UUIDs. Our attack is possible
even though each of the above sequences are included in just one document in
the training data.
We comprehensively evaluate our extraction attack to understand the factors
that contribute to its success. Worryingly, we find that larger models are more
vulnerable than smaller models. We conclude by drawing lessons and discussing
possible safeguards for training large language models.

本研究论文在大规模语言模型训练及私人数据集保护之间发现了一定的矛盾，由此提出了一种通过查询语言模型进行训练数据提取的攻击方法，并以 GPT-2 为例证，能够成功地提取训练数据中的个人信息、代码等敏感信息，这也提示着训练数据的隐私和安全问题，需要进一步的技术防范措施。