This work adapts and studies the gradient-based Membership Inference Test (gMINT) to the classification of text based on LLMs. MINT is a general approach intended to determine if given data was used for training machine learning models, and this work focuses on its application to the domain of Natural Language Processing. Using gradient-based analysis, the MINT model identifies whether particular data samples were included during the language model training phase, addressing growing concerns about data privacy in machine learning. The method was evaluated in seven Transformer-based models and six datasets comprising over 2.5 million sentences, focusing on text classification tasks. Experimental results demonstrate MINTs robustness, achieving AUC scores between 85% and 99%, depending on data size and model architecture. These findings highlight MINTs potential as a scalable and reliable tool for auditing machine learning models, ensuring transparency, safeguarding sensitive data, and fostering ethical compliance in the deployment of AI/NLP technologies.

本研究针对基于大型语言模型的文本分类任务，采用并研究了基于梯度的成员推断测试（gMINT），解决了日益严重的数据隐私问题。通过对七种基于变换器的模型和六个包含超过250万句的文本数据集进行评估，结果表明gMINT在模型审计中具有99%的AUC得分，展示了其作为一款可扩展且可靠的工具的潜力，以确保透明性并保护敏感数据。

我的文本在你的AI模型中吗？基于梯度的成员推断测试应用于大型语言模型