Large language models (LLMs) have been proven capable of memorizing their
training data, which can be extracted through specifically designed prompts. As
the scale of datasets continues to grow, privacy risks arising from
memorization have attracted increasing attention. Quantifying language model
memorization helps evaluate potential privacy risks. However, prior works on
quantifying memorization require access to the precise original data or incur
substantial computational overhead, making it difficult for applications in
real-world language models. To this end, we propose a fine-grained,
entity-level definition to quantify memorization with conditions and metrics
closer to real-world scenarios. In addition, we also present an approach for
efficiently extracting sensitive entities from autoregressive language models.
We conduct extensive experiments based on the proposed, probing language
models' ability to reconstruct sensitive entities under different settings. We
find that language models have strong memorization at the entity level and are
able to reproduce the training data even with partial leakages. The results
demonstrate that LLMs not only memorize their training data but also understand
associations between entities. These findings necessitate that trainers of LLMs
exercise greater prudence regarding model memorization, adopting memorization
mitigation techniques to preclude privacy violations.

大型语言模型通过精心设计的提示可以提取出其训练数据，这也带来了隐私风险。为了评估潜在的隐私风险，需要量化语言模型的记忆能力。本文提出了一个细粒度、基于实体级别的定义来量化记忆，并提供了一种从自回归语言模型中高效提取敏感实体的方法。实验结果显示，语言模型在实体级别上具有较强的记忆能力，并能够在部分泄露情况下重新生成训练数据。这些发现要求语言模型的训练者在模型记忆方面更加谨慎，采用记忆减轻技术以防止隐私侵犯。