Models of various NLP tasks have been shown to exhibit stereotypes, and the
bias in the question answering (QA) models is especially harmful as the output
answers might be directly consumed by the end users. There have been datasets
to evaluate bias in QA models, while bias mitigation technique for the QA
models is still under-explored. In this work, we propose BMBI, an approach to
mitigate the bias of multiple-choice QA models. Based on the intuition that a
model would lean to be more biased if it learns from a biased example, we
measure the bias level of a query instance by observing its influence on
another instance. If the influenced instance is more biased, we derive that the
query instance is biased. We then use the bias level detected as an
optimization objective to form a multi-task learning setting in addition to the
original QA task. We further introduce a new bias evaluation metric to quantify
bias in a comprehensive and sensitive way. We show that our method could be
applied to multiple QA formulations across multiple bias categories. It can
significantly reduce the bias level in all 9 bias categories in the BBQ dataset
while maintaining comparable QA accuracy.

提出了一种名为 BMBI 的方法，用于减轻多选问题回答模型中的偏见。通过观察一个查询实例对另一个实例的影响来度量查询实例的偏见水平，并将其作为优化目标，形成一个多任务学习设置。引入了一个新的偏见评估度量方法，以全面而敏感的方式量化偏见。实验证明，该方法可应用于多个偏见类别的多个问题回答公式，在不降低问题回答准确性的情况下显著降低了 BBQ 数据集中的 9 个偏见类别的偏见水平。