While state-of-the-art Large Language Models (LLMs) have shown impressive
performance on many tasks, there has been extensive research on undesirable
model behavior such as hallucinations and bias. In this work, we investigate
how the quality of LLM responses changes in terms of information accuracy,
truthfulness, and refusals depending on three user traits: English proficiency,
education level, and country of origin. We present extensive experimentation on
three state-of-the-art LLMs and two different datasets targeting truthfulness
and factuality. Our findings suggest that undesirable behaviors in
state-of-the-art LLMs occur disproportionately more for users with lower
English proficiency, of lower education status, and originating from outside
the US, rendering these models unreliable sources of information towards their
most vulnerable users.

通过对三种领先的大型语言模型 (LLM) 和两个不同的针对真实性和事实性的数据集进行深入实验，我们研究了 LLM 响应质量在信息准确性、真实性和拒绝方面如何随用户的英语水平、教育水平和国籍的不同而变化。我们的研究结果表明，领先的 LLM 模型在不可取的行为方面存在不均衡，对英语水平较低、教育水平较低和来自美国以外的用户具有更多的不可取行为，从而使这些模型对于最弱势的用户成为不可靠的信息来源。