The question-answering (QA) capabilities of foundation models are highly
sensitive to prompt variations, rendering their performance susceptible to
superficial, non-meaning-altering changes. This vulnerability often stems from
the model's preference or bias towards specific input characteristics, such as
option position or superficial image features in multi-modal settings. We
propose to rectify this bias directly in the model's internal representation.
Our approach, SteerFair, finds the bias direction in the model's representation
space and steers activation values away from it during inference. Specifically,
we exploit the observation that bias often adheres to simple association rules,
such as the spurious association between the first option and correctness
likelihood. Next, we construct demonstrations of these rules from unlabeled
samples and use them to identify the bias directions. We empirically show that
SteerFair significantly reduces instruction-tuned model performance variance
across prompt modifications on three benchmark tasks. Remarkably, our approach
surpasses a supervised baseline with 100 labels by an average of 10.86%
accuracy points and 12.95 score points and matches the performance with 500
labels.

我们的方法 SteerFair 通过找到模型表示空间中的偏见方向，并在推理过程中将激活值从偏见方向上移开，从而显著减少了提示变化对模型性能的影响，超过了 100 个标签的监督基准，平均准确率提高了 10.86％，分数提高了 12.95，并与 500 个标签的基准性能相匹配。

发现潜在空间中的偏倚：一种无监督的去偏方法

Discovering Bias in Latent Space: An Unsupervised Debiasing Approach

The ability of large language models (LLMs) to "learn in context" based on
the provided prompt has led to an explosive growth in their use, culminating in
the proliferation of AI assistants such as ChatGPT, Claude, and Bard. These AI
assistants are known to be robust to minor prompt modifications, mostly due to
alignment techniques that use human feedback. In contrast, the underlying
pre-trained LLMs they use as a backbone are known to be brittle in this
respect. Building high-quality backbone models remains a core challenge, and a
common approach to assessing their quality is to conduct few-shot evaluation.
Such evaluation is notorious for being highly sensitive to minor prompt
modifications, as well as the choice of specific in-context examples. Prior
work has examined how modifying different elements of the prompt can affect
model performance. However, these earlier studies tended to concentrate on a
limited number of specific prompt attributes and often produced contradictory
results. Additionally, previous research either focused on models with fewer
than 15 billion parameters or exclusively examined black-box models like GPT-3
or PaLM, making replication challenging. In the present study, we decompose the
entire prompt into four components: task description, demonstration inputs,
labels, and inline instructions provided for each demonstration. We investigate
the effects of structural and semantic corruptions of these elements on model
performance. We study models ranging from 1.5B to 70B in size, using ten
datasets covering classification and generation tasks. We find that repeating
text within the prompt boosts model performance, and bigger models ($\geq$30B)
are more sensitive to the semantics of the prompt. Finally, we observe that
adding task and inline instructions to the demonstrations enhances model
performance even when the instructions are semantically corrupted.

通过对大语言模型进行随机组成的实验，研究了任务描述、示例输入、标签、行内指令等多个因素对模型性能的影响，发现重复文本、大型模型以及任务和行内指令对模型性能提升具有积极影响。