The proliferation of large language models has revolutionized natural language processing tasks, yet it raises profound concerns regarding data privacy and security. Language models are trained on extensive corpora including potentially sensitive or proprietary information, and the risk of data leakage -- where the model response reveals pieces of such information -- remains inadequately understood. This study examines susceptibility to data leakage by quantifying the phenomenon of memorization in machine learning models, focusing on the evolution of memorization patterns over training. We investigate how the statistical characteristics of training data influence the memories encoded within the model by evaluating how repetition influences memorization. We reproduce findings that the probability of memorizing a sequence scales logarithmically with the number of times it is present in the data. Furthermore, we find that sequences which are not apparently memorized after the first encounter can be uncovered throughout the course of training even without subsequent encounters. The presence of these latent memorized sequences presents a challenge for data privacy since they may be hidden at the final checkpoint of the model. To this end, we develop a diagnostic test for uncovering these latent memorized sequences by considering their cross entropy loss.

该研究通过评估训练数据的统计特征对模型中的记忆编码产生的影响，重现了重复次数对记忆序列遗忘概率的对数标度关系，并发现即使没有后续接触，经过多次训练的数据仍然可以在训练过程中被揭示。由于这些潜在的记忆序列可能隐藏在模型的最终检查点上，这对数据隐私具有挑战性。为此，我们开发了一种通过考虑交叉熵损失来揭示这些潜在记忆序列的诊断测试。

揭开潜在记忆：评估大型语言模型中的数据泄露和记忆模式