Individuals and businesses have been significantly benefited by Large
Language Models (LLMs) including PaLM, Gemini and ChatGPT in various ways. For
example, LLMs enhance productivity, reduce costs, and enable us to focus on
more valuable tasks. Furthermore, LLMs possess the capacity to sift through
extensive datasets, uncover underlying patterns, and furnish critical insights
that propel the frontiers of technology and science. However, LLMs also pose
privacy concerns. Users' interactions with LLMs may expose their sensitive
personal or company information. A lack of robust privacy safeguards and legal
frameworks could permit the unwarranted intrusion or improper handling of
individual data, thereby risking infringements of privacy and the theft of
personal identities. To ensure privacy, it is essential to minimize the
dependency between shared prompts and private information. Various
randomization approaches have been proposed to protect prompts' privacy, but
they may incur utility loss compared to unprotected LLMs prompting. Therefore,
it is essential to evaluate the balance between the risk of privacy leakage and
loss of utility when conducting effective protection mechanisms. The current
study develops a framework for inferring privacy-protected Large Language
Models (LLMs) and lays down a solid theoretical basis for examining the
interplay between privacy preservation and utility. The core insight is
encapsulated within a theorem that is called as the NFL (abbreviation of the
word No-Free-Lunch) Theorem.

通过开发一个推断隐私保护的大型语言模型框架并提出无免费午餐（NFL）定理，本研究为检验隐私保护和效用之间的相互作用奠定了坚实的理论基础。

隐私保护条件下的逻辑回归模型推断的无免费午餐定理

No Free Lunch Theorem for Privacy-Preserving LLM Inference

We explore machine unlearning (MU) in the domain of large language models
(LLMs), referred to as LLM unlearning. This initiative aims to eliminate
undesirable data influence (e.g., sensitive or illegal information) and the
associated model capabilities, while maintaining the integrity of essential
knowledge generation and not affecting causally unrelated information. We
envision LLM unlearning becoming a pivotal element in the life-cycle management
of LLMs, potentially standing as an essential foundation for developing
generative AI that is not only safe, secure, and trustworthy, but also
resource-efficient without the need of full retraining. We navigate the
unlearning landscape in LLMs from conceptual formulation, methodologies,
metrics, and applications. In particular, we highlight the often-overlooked
aspects of existing LLM unlearning research, e.g., unlearning scope, data-model
interaction, and multifaceted efficacy assessment. We also draw connections
between LLM unlearning and related areas such as model editing, influence
functions, model explanation, adversarial training, and reinforcement learning.
Furthermore, we outline an effective assessment framework for LLM unlearning
and explore its applications in copyright and privacy safeguards and
sociotechnical harm reduction.

探索机器遗忘（MU）在大型语言模型（LLMs）领域中的应用，称为 LLM 遗忘。通过维持基本知识生成的完整性且不影响非因果相关信息，旨在消除不良数据影响（例如敏感或非法信息）及相关模型能力，成为 LLMs 生命周期管理中的重要元素，潜在地作为开发生成式 AI 的基础，既安全、安全、可信，又无需完全重新训练的资源高效方法。从概念制定、方法论、度量和应用等方面研究 LLMs 遗忘领域。重点突出现有 LLM 遗忘研究中常被忽视的方面，例如遗忘范围、数据模型交互和多方面功效评估。并在 LLM 遗忘与模型编辑、影响函数、模型解释、对抗训练和强化学习等相关领域之间建立联系。此外，我们还概述了 LLM 遗忘的有效评估框架，并探讨了其在版权和隐私保护以及降低社会技术危害方面的应用。

大型语言模型的机器遗忘思考

Rethinking Machine Unlearning for Large Language Models

A digital health twin can be defined as a virtual model of a physical person,
in this specific case, a patient. This virtual model is constituted by
multidimensional data that can host from clinical, molecular and therapeutic
parameters to sensor data and living conditions. Given that in computational
pathology, it is very important to have the information from image donors to
create computational models, the integration of digital twins in this field
could be crucial. However, since these virtual entities collect sensitive data
from physical people, privacy safeguards must also be considered and
implemented. With these data safeguards in place, health digital twins could
integrate digital clinical trials and be necessary participants in the
generation of real-world evidence, which could positively change both fields.

数字健康孪生可以定义为物理人的虚拟模型，由临床、分子、治疗参数、传感器数据和生活条件组成；与计算病理学相结合，有助于创建计算模型；数字临床试验的集成，必须考虑隐私保护措施。