We are currently in an era of fierce competition among various large language
models (LLMs) continuously pushing the boundaries of benchmark performance.
However, genuinely assessing the capabilities of these LLMs has become a
challenging and critical issue due to potential data contamination, and it
wastes dozens of time and effort for researchers and engineers to download and
try those contaminated models. To save our precious time, we propose a novel
and useful method, Clean-Eval, which mitigates the issue of data contamination
and evaluates the LLMs in a cleaner manner. Clean-Eval employs an LLM to
paraphrase and back-translate the contaminated data into a candidate set,
generating expressions with the same meaning but in different surface forms. A
semantic detector is then used to filter the generated low-quality samples to
narrow down this candidate set. The best candidate is finally selected from
this set based on the BLEURT score. According to human assessment, this best
candidate is semantically similar to the original contamination data but
expressed differently. All candidates can form a new benchmark to evaluate the
model. Our experiments illustrate that Clean-Eval substantially restores the
actual evaluation results on contaminated LLMs under both few-shot learning and
fine-tuning scenarios.

为了解决大型语言模型中的数据污染和评估问题，我们提出了一种名为 Clean-Eval 的方法，利用语义检测器筛选出生成的低质量样本，从候选集中选择出与原始污染数据在语义上相似但表达不同的最佳候选来形成一个新的基准，从而在少样本学习和微调场景下还原了受污染的语言模型的实际评估结果。