When querying a large language model (LLM), the context, i.e. personal,
demographic, and cultural information specific to an end-user, can
significantly shape the response of the LLM. For example, asking the model to
explain Newton's second law with the context "I am a toddler" yields a
different answer compared to the context "I am a physics professor." Proper
usage of the context enables the LLM to generate personalized responses,
whereas inappropriate contextual influence can lead to stereotypical and
potentially harmful generations (e.g. associating "female" with "housekeeper").
In practice, striking the right balance when leveraging context is a nuanced
and challenging problem that is often situation-dependent. One common approach
to address this challenge is to fine-tune LLMs on contextually appropriate
responses. However, this approach is expensive, time-consuming, and not
controllable for end-users in different situations. In this work, we propose
Context Steering (CoS) - a simple training-free method that can be easily
applied to autoregressive LLMs at inference time. By measuring the contextual
influence in terms of token prediction likelihood and modulating it, our method
enables practitioners to determine the appropriate level of contextual
influence based on their specific use case and end-user base. We showcase a
variety of applications of CoS including amplifying the contextual influence to
achieve better personalization and mitigating unwanted influence for reducing
model bias. In addition, we show that we can combine CoS with Bayesian
Inference to quantify the extent of hate speech on the internet. We demonstrate
the effectiveness of CoS on state-of-the-art LLMs and benchmarks.

通过测量预测标记的可能性以及调控它，我们提出了适用于自动回归 LL 模型的简单无需训练的方法，被称为 CoS （Context Steering），使实践者能够根据其特定用例和终端用户群确定适当的上下文影响水平，并展示了 CoS 的各种应用，包括增强上下文影响以实现更好的个性化和减少模型偏差所产生的无用影响，此外，还展示了如何结合贝叶斯推断来量化互联网上的仇恨言论，我们在最先进的 LL 模型和基准测试上展示了 CoS 的有效性。

通过上下文引导增强个性化并减少偏见的 CoS

CoS: Enhancing Personalization and Mitigating Bias with Context Steering

This paper provides new theory to support to the eXplainable AI (XAI) method
Contextual Importance and Utility (CIU). CIU arithmetic is based on the
concepts of Multi-Attribute Utility Theory, which gives CIU a solid theoretical
foundation. The novel concept of contextual influence is also defined, which
makes it possible to compare CIU directly with so-called additive feature
attribution (AFA) methods for model-agnostic outcome explanation. One key
takeaway is that the "influence" concept used by AFA methods is inadequate for
outcome explanation purposes even for simple models to explain. Experiments
with simple models show that explanations using contextual importance (CI) and
contextual utility (CU) produce explanations where influence-based methods
fail. It is also shown that CI and CU guarantees explanation faithfulness
towards the explained model.

本文提出了一个新理论，支持可解释 AI（XAI）方法的情境重要性和效用（CIU）。CIU 算法基于多属性效用理论的概念，为 CIU 提供了坚实的理论基础。同时，也定义了新颖的情境影响概念，使 CIU 能够直接与模型无关的加法特征归因方法进行比较。实验表明，使用 CI 和 CU 的解释可以产生 CIU 方法解释失败的影响方法解释。此外，还表明 CI 和 CU 保证向解释模型的说明忠实。