With the rise of Large Language Models (LLMs) in recent years, new
opportunities are emerging, but also new challenges, and contamination is
quickly becoming critical. Business applications and fundraising in AI have
reached a scale at which a few percentage points gained on popular
question-answering benchmarks could translate into dozens of millions of
dollars, placing high pressure on model integrity. At the same time, it is
becoming harder and harder to keep track of the data that LLMs have seen; if
not impossible with closed-source models like GPT-4 and Claude-3 not divulging
any information on the training set. As a result, contamination becomes a
critical issue: LLMs' performance may not be reliable anymore, as the high
performance may be at least partly due to their previous exposure to the data.
This limitation jeopardizes the entire progress in the field of NLP, yet, there
remains a lack of methods on how to efficiently address contamination, or a
clear consensus on prevention, mitigation and classification of contamination.
In this paper, we survey all recent work on contamination with LLMs, and help
the community track contamination levels of LLMs by releasing an open-source
Python library named LLMSanitize implementing major contamination detection
algorithms, which link is: this https URL

LLMs 应用的规模不断扩大，但同时也带来了污染问题，而模型的完整性对业务应用和筹款至关重要。本文调查了 LLMs 污染问题的最新研究，并通过开源 Python 库 LLMSanitize 实现了主要污染检测算法，以帮助社区跟踪 LLMs 的污染水平。