Machine learning practitioners often fine-tune generative pre-trained models
like GPT-3 to improve model performance at specific tasks. Previous works,
however, suggest that fine-tuned machine learning models memorize and emit
sensitive information from the original fine-tuning dataset. Companies such as
OpenAI offer fine-tuning services for their models, but no prior work has
conducted a memorization attack on any closed-source models. In this work, we
simulate a privacy attack on GPT-3 using OpenAI's fine-tuning API. Our
objective is to determine if personally identifiable information (PII) can be
extracted from this model. We (1) explore the use of naive prompting methods on
a GPT-3 fine-tuned classification model, and (2) we design a practical word
generation task called Autocomplete to investigate the extent of PII
memorization in fine-tuned GPT-3 within a real-world context. Our findings
reveal that fine-tuning GPT3 for both tasks led to the model memorizing and
disclosing critical personally identifiable information (PII) obtained from the
underlying fine-tuning dataset. To encourage further research, we have made our
codes and datasets publicly available on GitHub at:
this https URL

本研究使用 OpenAI 的 fine-tuning API 对 GPT-3 进行隐私攻击，以确定是否可以从该模型中提取出个人身份信息（PII）。发现在针对两种任务的 fine-tuning GPT-3 中，模型会记忆和泄露从底层 fine-tuning 数据集中获取的关键的个人身份信息。

使用 OpenAI API 来微调 GPT-3 是否会泄露个人身份信息？

Does fine-tuning GPT-3 with the OpenAI API leak personally-identifiable  information?

The newly emerged machine learning (e.g. deep learning) methods have become a
strong driving force to revolutionize a wide range of industries, such as smart
healthcare, financial technology, and surveillance systems. Meanwhile, privacy
has emerged as a big concern in this machine learning-based artificial
intelligence era. It is important to note that the problem of privacy
preservation in the context of machine learning is quite different from that in
traditional data privacy protection, as machine learning can act as both friend
and foe. Currently, the work on the preservation of privacy and machine
learning (ML) is still in an infancy stage, as most existing solutions only
focus on privacy problems during the machine learning process. Therefore, a
comprehensive study on the privacy preservation problems and machine learning
is required. This paper surveys the state of the art in privacy issues and
solutions for machine learning. The survey covers three categories of
interactions between privacy and machine learning: (i) private machine
learning, (ii) machine learning aided privacy protection, and (iii) machine
learning-based privacy attack and corresponding protection schemes. The current
research progress in each category is reviewed and the key challenges are
identified. Finally, based on our in-depth analysis of the area of privacy and
machine learning, we point out future research directions in this field.

本文对机器学习与隐私问题进行了综述，涵盖了三种交互方式：(i) 私有机器学习，(ii) 机器学习辅助隐私保护，以及 (iii) 基于机器学习的隐私攻击和相应的保护方案，总结了各个领域内的研究进展和关键挑战，并提出了未来的研究方向。

当机器学习遇到隐私：一份调查与展望

When Machine Learning Meets Privacy: A Survey and Outlook

Many 3D vision systems localize cameras within a scene using 3D point clouds.
Such point clouds are often obtained using structure from motion (SfM), after
which the images are discarded to preserve privacy. In this paper, we show, for
the first time, that such point clouds retain enough information to reveal
scene appearance and compromise privacy. We present a privacy attack that
reconstructs color images of the scene from the point cloud. Our method is
based on a cascaded U-Net that takes as input, a 2D multichannel image of the
points rendered from a specific viewpoint containing point depth and optionally
color and SIFT descriptors and outputs a color image of the scene from that
viewpoint. Unlike previous feature inversion methods, we deal with highly
sparse and irregular 2D point distributions and inputs where many point
attributes are missing, namely keypoint orientation and scale, the descriptor
image source and the 3D point visibility. We evaluate our attack algorithm on
public datasets and analyze the significance of the point cloud attributes.
Finally, we show that novel views can also be generated thereby enabling
compelling virtual tours of the underlying scene.

通过结构运动（SfM）算法获取的点云中的信息足以泄露场景外貌并危及隐私，同时使用级联 U-Net 方法可以重构场景的彩色图像，并生成新的视图，为场景的虚拟旅游提供了可能。