Large language models (LLMs) often make factually incorrect responses despite
their success in various applications. In this paper, we hypothesize that
relying heavily on simple co-occurrence statistics of the pre-training corpora
is one of the main factors that cause factual errors. Our results reveal that
LLMs are vulnerable to the co-occurrence bias, defined as preferring frequently
co-occurred words over the correct answer. Consequently, LLMs struggle to
recall facts whose subject and object rarely co-occur in the pre-training
dataset although they are seen during finetuning. We show that co-occurrence
bias remains despite scaling up model sizes or finetuning. Therefore, we
suggest finetuning on a debiased dataset to mitigate the bias by filtering out
biased samples whose subject-object co-occurrence count is high. Although
debiased finetuning allows LLMs to memorize rare facts in the training set, it
is not effective in recalling rare facts unseen during finetuning. Further
research in mitigation will help build reliable language models by preventing
potential errors. The code is available at
https://github.com/CheongWoong/impact_of_cooccurrence.

大型语言模型在各种应用中取得成功，但往往会出现事实性错误。本文研究了共现偏差对语言模型造成的影响，并提供了一种通过在去偏数据集上微调来减轻偏差的方法。

大型语言模型中共现对事实知识的影响

Impact of Co-occurrence on Factual Knowledge of Large Language Models

Contextual information is a valuable cue for Deep Neural Networks (DNNs) to
learn better representations and improve accuracy. However, co-occurrence bias
in the training dataset may hamper a DNN model's generalizability to unseen
scenarios in the real world. For example, in COCO, many object categories have
a much higher co-occurrence with men compared to women, which can bias a DNN's
prediction in favor of men. Recent works have focused on task-specific training
strategies to handle bias in such scenarios, but fixing the available data is
often ignored. In this paper, we propose a novel and more generic solution to
address the contextual bias in the datasets by selecting a subset of the
samples, which is fair in terms of the co-occurrence with various classes for a
protected attribute. We introduce a data repair algorithm using the coefficient
of variation, which can curate fair and contextually balanced data for a
protected class(es). This helps in training a fair model irrespective of the
task, architecture or training methodology. Our proposed solution is simple,
effective, and can even be used in an active learning setting where the data
labels are not present or being generated incrementally. We demonstrate the
effectiveness of our algorithm for the task of object detection and multi-label
image classification across different datasets. Through a series of
experiments, we validate that curating contextually fair data helps make model
predictions fair by balancing the true positive rate for the protected class
across groups without compromising on the model's overall performance.

本文提出了一种利用数据修复算法解决数据集中的语境偏差问题的解决方案，该算法可以通过平衡受保护属性的各种类别的共同出现率来筛选样本，从而训练出公平的模型，而不会牺牲模型整体性能。

数据修复是否能实现公平模型？通过筛选上下文公平数据减少模型偏差

Does Data Repair Lead to Fair Models? Curating Contextually Fair Data To  Reduce Model Bias

Deep neural networks (DNNs) have a high accuracy on image classification
tasks. However, DNNs trained by such dataset with co-occurrence bias may rely
on wrong features while making decisions for classification. It will greatly
affect the transferability of pre-trained DNNs. In this paper, we propose an
interactive method to direct classifiers paying attentions to the regions that
are manually specified by the users, in order to mitigate the influence of
co-occurrence bias. We test on CelebA dataset, the pre-trained AlexNet is
fine-tuned to focus on the specific facial attributes based on the results of
Grad-CAM.

本文提出一种交互式的方法，通过手动指定区域，让分类器注意到这些区域，以减轻共现偏差对预训练深度神经网络的影响，在 CelebA 数据集上测试并 fine-tuned 预训练 AlexNet 以关注指定的面部属性。