The era post-2018 marked the advent of Large Language Models (LLMs), with
innovations such as OpenAI's ChatGPT showcasing prodigious linguistic prowess.
As the industry galloped toward augmenting model parameters and capitalizing on
vast swaths of human language data, security and privacy challenges also
emerged. Foremost among these is the potential inadvertent accrual of Personal
Identifiable Information (PII) during web-based data acquisition, posing risks
of unintended PII disclosure. While strategies like RLHF during training and
Catastrophic Forgetting have been marshaled to control the risk of privacy
infringements, recent advancements in LLMs, epitomized by OpenAI's fine-tuning
interface for GPT-3.5, have reignited concerns. One may ask: can the
fine-tuning of LLMs precipitate the leakage of personal information embedded
within training datasets? This paper reports the first endeavor to seek the
answer to the question, particularly our discovery of a new LLM exploitation
avenue, called the Janus attack. In the attack, one can construct a PII
association task, whereby an LLM is fine-tuned using a minuscule PII dataset,
to potentially reinstate and reveal concealed PIIs. Our findings indicate that,
with a trivial fine-tuning outlay, LLMs such as GPT-3.5 can transition from
being impermeable to PII extraction to a state where they divulge a substantial
proportion of concealed PII. This research, through its deep dive into the
Janus attack vector, underscores the imperative of navigating the intricate
interplay between LLM utility and privacy preservation.

利用微小的个人可识别信息数据集完成 GPT-3.5 的微调，可以导致大量隐藏的个人信息被揭示。