Entity bias widely affects pretrained (large) language models, causing them
to excessively rely on (biased) parametric knowledge to make unfaithful
predictions. Although causality-inspired methods have shown great potential to
mitigate entity bias, it is hard to precisely estimate the parameters of
underlying causal models in practice. The rise of black-box LLMs also makes the
situation even worse, because of their inaccessible parameters and uncalibrated
logits. To address these problems, we propose a specific structured causal
model (SCM) whose parameters are comparatively easier to estimate. Building
upon this SCM, we propose causal intervention techniques to mitigate entity
bias for both white-box and black-box settings. The proposed causal
intervention perturbs the original entity with neighboring entities. This
intervention reduces specific biasing information pertaining to the original
entity while still preserving sufficient common predictive information from
similar entities. When evaluated on the relation extraction task, our
training-time intervention significantly improves the F1 score of RoBERTa by
5.7 points on EntRED, in which spurious shortcuts between entities and labels
are removed. Meanwhile, our in-context intervention effectively reduces the
knowledge conflicts between parametric knowledge and contextual knowledge in
GPT-3.5 and improves the F1 score by 9.14 points on a challenging test set
derived from Re-TACRED.

通过提出特定的结构因果模型（SCM）及其上的因果干预技术，在白盒和黑盒模型中缓解了实体偏差，该干预通过干扰原始实体与邻近实体来减少与原始实体有关的特定偏见信息，同时仍保留来自相似实体的充分共同预测信息，在关系抽取任务上，我们的训练时间干预将 RoBERTa 的 F1 得分在 EntRED 上提高了 5.7 分，在 GPT-3.5 上，我们的上下文干预有效减少了参数化知识和上下文知识之间的知识冲突，并将 F1 得分在一个具有挑战性的测试集中提高了 9.14 分。