We present a large-scale study of linguistic bias exhibited by ChatGPT
covering ten dialects of English (Standard American English, Standard British
English, and eight widely spoken non-"standard" varieties from around the
world). We prompted GPT-3.5 Turbo and GPT-4 with text by native speakers of
each variety and analyzed the responses via detailed linguistic feature
annotation and native speaker evaluation. We find that the models default to
"standard" varieties of English; based on evaluation by native speakers, we
also find that model responses to non-"standard" varieties consistently exhibit
a range of issues: lack of comprehension (10% worse compared to "standard"
varieties), stereotyping (16% worse), demeaning content (22% worse), and
condescending responses (12% worse). We also find that if these models are
asked to imitate the writing style of prompts in non-"standard" varieties, they
produce text that exhibits lower comprehension of the input and is especially
prone to stereotyping. GPT-4 improves on GPT-3.5 in terms of comprehension,
warmth, and friendliness, but it also results in a marked increase in
stereotyping (+17%). The results suggest that GPT-3.5 Turbo and GPT-4 exhibit
linguistic discrimination in ways that can exacerbate harms for speakers of
non-"standard" varieties.

GPT-3.5 Turbo 和 GPT-4 对非 “标准” 语言变体表现出语言偏见，导致理解能力减弱、刻板印象加剧，并存在辱人内容和居高临下的回应，这对非 “标准” 语言使用者产生了潜在的伤害。