Despite the strong capabilities of Large Language Models (LLMs) to acquire
knowledge from their training corpora, the memorization of sensitive
information in the corpora such as copyrighted, harmful, and private content
has led to ethical and legal concerns. In response to these challenges,
unlearning has emerged as a potential remedy for LLMs affected by problematic
training data. However, previous unlearning techniques are either not
applicable to black-box LLMs due to required access to model internal weights,
or violate data protection principles by retaining sensitive data for
inference-time correction. We propose $\delta$-unlearning, an offset unlearning
framework for black-box LLMs. Instead of tuning the black-box LLM itself,
$\delta$-unlearning learns the logit offset needed for unlearning by
contrasting the logits from a pair of smaller models. Experiments demonstrate
that $\delta$-unlearning can effectively unlearn target data while maintaining
similar or even stronger performance on general out-of-forget-scope tasks.
$\delta$-unlearning also effectively incorporates different unlearning
algorithms, making our approach a versatile solution to adapting various
existing unlearning algorithms to black-box LLMs.

用于黑盒大型语言模型的偏移学习框架，通过对比一对较小模型的逻辑斯谛，有效地遗忘目标数据，同时在一般范围任务上保持类似或甚至更强的性能，并有效地整合不同的遗忘算法。

大型语言模型的位移遗忘

Offset Unlearning for Large Language Models

Adapting state-of-the-art Large Language Models (LLMs) like GPT-4 and Gemini
for specific tasks is challenging. Due to the opacity in their parameters,
embeddings, and even output probabilities, existing fine-tuning adaptation
methods are inapplicable. Consequently, adapting these black-box LLMs is only
possible through their API services, raising concerns about transparency,
privacy, and cost. To address these challenges, we introduce BBox-Adapter, a
novel lightweight adapter for black-box LLMs. BBox-Adapter distinguishes target
and source domain data by treating target data as positive and source data as
negative. It employs a ranking-based Noise Contrastive Estimation (NCE) loss to
promote the likelihood of target domain data while penalizing that of the
source domain. Furthermore, it features an online adaptation mechanism, which
incorporates real-time positive data sampling from ground-truth, human, or AI
feedback, coupled with negative data from previous adaptations. Extensive
experiments demonstrate BBox-Adapter's effectiveness and cost efficiency. It
improves model performance by up to 6.77% across diverse tasks and domains,
while reducing training and inference costs by 31.30x and 1.84x, respectively.

使用 BBox-Adapter 对黑盒大型语言模型进行轻量级适配，通过实时正数据采样和先前适配的负数据，提高了模型性能，降低了训练和推理成本。

BBox-Adapter：针对黑盒大型语言模型的轻量级适配

BBox-Adapter: Lightweight Adapting for Black-Box Large Language Models

Large language models (LLMs) specializing in natural language generation
(NLG) have recently started exhibiting promising capabilities across a variety
of domains. However, gauging the trustworthiness of responses generated by LLMs
remains an open challenge, with limited research on uncertainty quantification
for NLG. Furthermore, existing literature typically assumes white-box access to
language models, which is becoming unrealistic either due to the closed-source
nature of the latest LLMs or due to computational constraints. In this work, we
investigate uncertainty quantification in NLG for $\textit{black-box}$ LLMs. We
first differentiate two closely-related notions: $\textit{uncertainty}$, which
depends only on the input, and $\textit{confidence}$, which additionally
depends on the generated response. We then propose and compare several
confidence/uncertainty metrics, applying them to $\textit{selective NLG}$,
where unreliable results could either be ignored or yielded for further
assessment. Our findings on several popular LLMs and datasets reveal that a
simple yet effective metric for the average semantic dispersion can be a
reliable predictor of the quality of LLM responses. This study can provide
valuable insights for practitioners on uncertainty management when adopting
LLMs. The code to replicate all our experiments is available at
this https URL

该研究探讨了基于黑盒 LLMs 的自然语言生成的不确定性计量，提出几个置信度 / 不确定度统计度量标准，并发现语义分散的平均值可以作为评估 LLMs 响应质量的可靠指标。