Named geographic entities (geo-entities for short) are the building blocks of
many geographic datasets. Characterizing geo-entities is integral to various
application domains, such as geo-intelligence and map comprehension, while a
key challenge is to capture the spatial-varying context of an entity. We
hypothesize that we shall know the characteristics of a geo-entity by its
surrounding entities, similar to knowing word meanings by their linguistic
context. Accordingly, we propose a novel spatial language model, SpaBERT, which
provides a general-purpose geo-entity representation based on neighboring
entities in geospatial data. SpaBERT extends BERT to capture linearized spatial
context, while incorporating a spatial coordinate embedding mechanism to
preserve spatial relations of entities in the 2-dimensional space. SpaBERT is
pretrained with masked language modeling and masked entity prediction tasks to
learn spatial dependencies. We apply SpaBERT to two downstream tasks:
geo-entity typing and geo-entity linking. Compared with the existing language
models that do not use spatial context, SpaBERT shows significant performance
improvement on both tasks. We also analyze the entity representation from
SpaBERT in various settings and the effect of spatial coordinate embedding.

本文提出了一种新颖的 SpaBERT 空间语言模型，使用邻近地理实体建立一种通用的地理实体表示，同时扩展了基于 Bert 模型的句子表示方法以及加入一个空间坐标嵌入机制，用于保留二维空间中实体之间的空间关系，结果表明 SpaBERT 在 geo-entity typing 和 geo-entity linking 两个任务上相比不使用空间上下文的现有语言模型表现有了显著的提升。