Large Language Models (LLMs) inherently carry the biases contained in their
training corpora, which can lead to the perpetuation of societal harm. As the
impact of these foundation models grows, understanding and evaluating their
biases becomes crucial to achieving fairness and accuracy. We propose to study
what LLMs know about the world we live in through the lens of geography. This
approach is particularly powerful as there is ground truth for the numerous
aspects of human life that are meaningfully projected onto geographic space
such as culture, race, language, politics, and religion. We show various
problematic geographic biases, which we define as systemic errors in geospatial
predictions. Initially, we demonstrate that LLMs are capable of making accurate
zero-shot geospatial predictions in the form of ratings that show strong
monotonic correlation with ground truth (Spearman's $\rho$ of up to 0.89). We
then show that LLMs exhibit common biases across a range of objective and
subjective topics. In particular, LLMs are clearly biased against locations
with lower socioeconomic conditions (e.g. most of Africa) on a variety of
sensitive subjective topics such as attractiveness, morality, and intelligence
(Spearman's $\rho$ of up to 0.70). Finally, we introduce a bias score to
quantify this and find that there is significant variation in the magnitude of
bias across existing LLMs.

通过地理学角度研究大型语言模型的地理知识，展示其在地理空间预测中存在的问题性地理偏差，并 quantifies 目前大型语言模型的偏差程度。