Fine-tuning is a common and effective method for tailoring large language
models (LLMs) to specialized tasks and applications. In this paper, we study
the privacy implications of fine-tuning LLMs on user data. To this end, we
define a realistic threat model, called user inference, wherein an attacker
infers whether or not a user's data was used for fine-tuning. We implement
attacks for this threat model that require only a small set of samples from a
user (possibly different from the samples used for training) and black-box
access to the fine-tuned LLM. We find that LLMs are susceptible to user
inference attacks across a variety of fine-tuning datasets, at times with near
perfect attack success rates. Further, we investigate which properties make
users vulnerable to user inference, finding that outlier users (i.e. those with
data distributions sufficiently different from other users) and users who
contribute large quantities of data are most susceptible to attack. Finally, we
explore several heuristics for mitigating privacy attacks. We find that
interventions in the training algorithm, such as batch or per-example gradient
clipping and early stopping fail to prevent user inference. However, limiting
the number of fine-tuning samples from a single user can reduce attack
effectiveness, albeit at the cost of reducing the total amount of fine-tuning
data.

研究表明，通过对用户数据进行细调的大型语言模型（LLMs）存在用户推测攻击的隐私风险，攻击者可以通过仅需少量用户样本和黑盒访问细调后的 LLMs 来推断用户的数据是否被用于细调，通过限制单个用户的细调样本数量可以减少攻击效果，但也会降低细调数据总量。