BriefGPT.xyz
Apr, 2024
模型崩溃是否不可避免?通过积累真实和合成数据打破递归的诅咒
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
HTML
PDF
Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey, Rafael Rafailov, Henry Sleight...
TL;DR
本文探讨了生成模型在其自身生成的输出上进行训练时可能导致的模型崩溃问题,并通过理论和实证研究表明数据的积累可以缓解模型崩溃的问题。
Abstract
The proliferation of
generative models
, combined with
pretraining
on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs? Recent investigations into
→