Past work has shown that large language models are susceptible to privacy
attacks, where adversaries generate sequences from a trained model and detect
which sequences are memorized from the training set. In this work, we show that
the success of these attacks is largely due to duplication in commonly used
web-scraped training sets. We first show that the rate at which language models
regenerate training sequences is superlinearly related to a sequence's count in
the training set. For instance, a sequence that is present 10 times in the
training data is on average generated ~1000 times more often than a sequence
that is present only once. We next show that existing methods for detecting
memorized sequences have near-chance accuracy on non-duplicated training
sequences. Finally, we find that after applying methods to deduplicate training
data, language models are considerably more secure against these types of
privacy attacks. Taken together, our results motivate an increased focus on
deduplication in privacy-sensitive applications and a reevaluation of the
practicality of existing privacy attacks.

该研究显示，大型语言模型在面对隐私攻击时，其攻击的成功与常用网络爬取的训练集中的重复数据有很大关系，而消除重复数据的方法可以显著提高语言模型的隐私安全性。