Despite the significant achievements of existing prompting methods such as
in-context learning and chain-of-thought for large language models (LLMs), they
still face challenges of various biases. Traditional debiasing methods
primarily focus on the model training stage, including data augmentation-based
and reweight-based approaches, with the limitations of addressing the complex
biases of LLMs. To address such limitations, the causal relationship behind the
prompting methods is uncovered using a structural causal model, and a novel
causal prompting method based on front-door adjustment is proposed to
effectively mitigate the bias of LLMs. In specific, causal intervention is
implemented by designing the prompts without accessing the parameters and
logits of LLMs.The chain-of-thoughts generated by LLMs are employed as the
mediator variable and the causal effect between the input prompt and the output
answers is calculated through front-door adjustment to mitigate model biases.
Moreover, to obtain the representation of the samples precisely and estimate
the causal effect more accurately, contrastive learning is used to fine-tune
the encoder of the samples by aligning the space of the encoder with the LLM.
Experimental results show that the proposed causal prompting approach achieves
excellent performance on 3 natural language processing datasets on both
open-source and closed-source LLMs.

通过揭示结构性因果模型的因果关系，提出了一种基于前门调整的新颖因果提示方法，通过设计不访问大型语言模型的参数和逻辑回归的提示，实施因果干预，并通过前门调整计算输入提示和输出答案之间的因果效应来减轻模型偏差。