This paper investigates the radioactivity of LLM-generated texts, i.e.
whether it is possible to detect that such input was used as training data.
Conventional methods like membership inference can carry out this detection
with some level of accuracy. We show that watermarked training data leaves
traces easier to detect and much more reliable than membership inference. We
link the contamination level to the watermark robustness, its proportion in the
training set, and the fine-tuning process. We notably demonstrate that training
on watermarked synthetic instructions can be detected with high confidence
(p-value < 1e-5) even when as little as 5% of training text is watermarked.
Thus, LLM watermarking, originally designed for detecting machine-generated
text, gives the ability to easily identify if the outputs of a watermarked LLM
were used to fine-tune another LLM.

调查了 LLM 生成的文本的辐射性，即是否可能检测到这种输入被用作训练数据；与成员推断等传统方法相比，我们发现水印训练数据留下的痕迹更容易检测且更可靠；我们将污染程度与水印的鲁棒性、在训练集中所占比例和微调过程联系起来；我们特别证明，即使仅有 5％的训练文本带有水印，也能以高置信度（p 值 < 1e-5）检测到在带有水印的合成指令上进行训练；因此，最初设计用于检测机器生成文本的 LLM 水印技术可以轻松识别是否使用带有水印的 LLM 的输出来进行微调。