Fine-tuning on task-specific question-answer pairs is a predominant method
for enhancing the performance of instruction-tuned large language models (LLMs)
on downstream tasks. However, in certain specialized domains, such as
healthcare or harmless content generation, it is nearly impossible to obtain a
large volume of high-quality data that matches the downstream distribution. To
improve the performance of LLMs in data-scarce domains with domain-mismatched
data, we re-evaluated the Transformer architecture and discovered that not all
parameter updates during fine-tuning contribute positively to downstream
performance. Our analysis reveals that within the self-attention and
feed-forward networks, only the fine-tuned attention parameters are
particularly beneficial when the training set's distribution does not fully
align with the test set. Based on this insight, we propose an effective
inference-time intervention method: \uline{T}raining \uline{A}ll parameters but
\uline{I}nferring with only \uline{A}ttention (\trainallInfAttn). We
empirically validate \trainallInfAttn using two general instruction-tuning
datasets and evaluate it on seven downstream tasks involving math, reasoning,
and knowledge understanding across LLMs of different parameter sizes and
fine-tuning techniques. Our comprehensive experiments demonstrate that
\trainallInfAttn achieves superior improvements compared to both the fully
fine-tuned model and the base model in most scenarios, with significant
performance gains. The high tolerance of \trainallInfAttn to data mismatches
makes it resistant to jailbreaking tuning and enhances specialized tasks using
general data.

通过细调参数和推理时干预的 Transformer 模型，能够在数据匮乏的领域中提高大语言模型的性能。

TAIA：大型语言模型的非分布式数据学习者

TAIA: Large Language Models are Out-of-Distribution Data Learners

We introduce Inference-Time Intervention (ITI), a technique designed to
enhance the truthfulness of large language models (LLMs). ITI operates by
shifting model activations during inference, following a set of directions
across a limited number of attention heads. This intervention significantly
improves the performance of LLaMA models on the TruthfulQA benchmark. On an
instruction-finetuned LLaMA called Alpaca, ITI improves its truthfulness from
32.5% to 65.1%. We identify a tradeoff between truthfulness and helpfulness and
demonstrate how to balance it by tuning the intervention strength. ITI is
minimally invasive and computationally inexpensive. Moreover, the technique is
data efficient: while approaches like RLHF require extensive annotations, ITI
locates truthful directions using only few hundred examples. Our findings
suggest that LLMs may have an internal representation of the likelihood of
something being true, even as they produce falsehoods on the surface.

介绍了一种名为 Inference-Time Intervention 的技术，它通过在有限数量的注意头上遵循一组指南，在推理期间改变模型的激活来提高大型语言模型的真实性，该技术在 TruthfulQA 基准测试中显著提高了 LLaMA 模型的性能。