This study examines the use of Large Language Models (LLMs) for retrieving
factual information, addressing concerns over their propensity to produce
factually incorrect "hallucinated" responses or to altogether decline to even
answer prompt at all. Specifically, it investigates the presence of
gender-based biases in LLMs' responses to factual inquiries. This paper takes a
multi-pronged approach to evaluating GPT models by evaluating fairness across
multiple dimensions of recall, hallucinations and declinations. Our findings
reveal discernible gender disparities in the responses generated by GPT-3.5.
While advancements in GPT-4 have led to improvements in performance, they have
not fully eradicated these gender disparities, notably in instances where
responses are declined. The study further explores the origins of these
disparities by examining the influence of gender associations in prompts and
the homogeneity in the responses.

该研究评估了大型语言模型对获取事实信息的使用，特别研究了这些模型在回答问题时可能产生错误回答或完全拒绝回答的倾向，并重点调查了性别差异在模型回答中的存在。研究结果显示 GPT-3.5 生成的回答存在明显性别差异，尽管 GPT-4 的进展改善了性能，但并未完全消除这些性别差异，特别是在回答被拒绝的情况下。研究进一步探讨了提示中性别相关性和回答的同质性对这些差异的影响。