Large Language Models (LLMs) are trained on large-scale web data, which makes
it difficult to grasp the contribution of each text. This poses the risk of
leaking inappropriate data such as benchmarks, personal information, and
copyrighted texts in the training data. Membership Inference Attacks (MIA),
which determine whether a given text is included in the model's training data,
have been attracting attention. Previous studies of MIAs revealed that
likelihood-based classification is effective for detecting leaks in LLMs.
However, the existing methods cannot be applied to some proprietary models like
ChatGPT or Claude 3 because the likelihood is unavailable to the user. In this
study, we propose a Sampling-based Pseudo-Likelihood (\textbf{SPL}) method for
MIA (\textbf{SaMIA}) that calculates SPL using only the text generated by an
LLM to detect leaks. The SaMIA treats the target text as the reference text and
multiple outputs from the LLM as text samples, calculates the degree of
$n$-gram match as SPL, and determines the membership of the text in the
training data. Even without likelihoods, SaMIA performed on par with existing
likelihood-based methods.

该研究提出了一种基于采样的伪似然方法（SaMIA），通过仅使用大型语言模型生成的文本来计算伪似然，以检测数据泄漏，即使没有似然度，SaMIA 的表现与现有的基于似然度的方法相媲美。