The training data for LLMs embeds societal values, increasing their familiarity with the language's culture. Our analysis found that 44% of the variance in the ability of GPT-4o to reflect the societal values of a country, as measured by the World Values Survey, correlates with the availability of digital resources in that language. Notably, the error rate was more than five times higher for the languages of the lowest resource compared to the languages of the highest resource. For GPT-4-turbo, this correlation rose to 72%, suggesting efforts to improve the familiarity with the non-English language beyond the web-scraped data. Our study developed one of the largest and most robust datasets in this topic area with 21 country-language pairs, each of which contain 94 survey questions verified by native speakers. Our results highlight the link between LLM performance and digital data availability in target languages. Weaker performance in low-resource languages, especially prominent in the Global South, may worsen digital divides. We discuss strategies proposed to address this, including developing multilingual LLMs from the ground up and enhancing fine-tuning on diverse linguistic datasets, as seen in African language initiatives.

本研究解决了大型语言模型（LLMs）在不同语言资源可用性下，反映国家社会价值观的能力差异问题。通过分析，我们发现数字资源的可用性对模型性能的影响显著，尤其是在低资源语言中性能下降明显，可能加剧数字鸿沟。研究提出了通过建立多语言LLMs和丰富多样的语言数据集进行微调的策略，以改善弱资源语言的表现。

大型语言模型中的文化忠诚度：在线语言资源对模型价值表现的影响评估