BriefGPT.xyz
Feb, 2025
偏好泄漏:LLM作为评审的污染问题
Preference Leakage: A Contamination Problem in LLM-as-a-judge
HTML
PDF
Dawei Li, Renliang Sun, Yue Huang, Ming Zhong, Bohan Jiang...
TL;DR
本研究关注LLM作为评审时可能出现的偏好泄漏污染问题,探讨了数据生成器与评审模型之间的相关性对结果的影响。通过定义三种相关性并进行广泛实验,本文揭示了偏好泄漏广泛存在且难以检测的特性,指出其对模型评估和训练的潜在负面影响。
Abstract
Large Language Models
(LLMs) as judges and LLM-based data synthesis have emerged as two fundamental LLM-driven
Data Annotation
methods in model development. While their combination significantly enhances the effi
→