Pretrained language models (PLMs) often fail to fairly represent target users
from certain world regions because of the under-representation of those regions
in training datasets. With recent PLMs trained on enormous data sources,
quantifying their potential biases is difficult, due to their black-box nature
and the sheer scale of the data sources. In this work, we devise an approach to
study the geographic bias (and knowledge) present in PLMs, proposing a
Geographic-Representation Probing Framework adopting a self-conditioning method
coupled with entity-country mappings. Our findings suggest PLMs'
representations map surprisingly well to the physical world in terms of
country-to-country associations, but this knowledge is unequally shared across
languages. Last, we explain how large PLMs despite exhibiting notions of
geographical proximity, over-amplify geopolitical favouritism at inference
time.

本文提出了一种地理表示探究框架，通过自我条件方法和实体 - 国家映射研究了 PLMs 中存在的地理偏见和知识，发现尽管 PLMs 的表示与物理世界的各个国家的关联性惊人地契合，但这种知识在不同语言中分享程度却有所不同，同时说明了大型 PLMs 虽然表现出了地理接近度的概念，但在推断时会过度放大地缘政治偏好。