We empirically demonstrate that a transformer pre-trained on country-scale
unlabeled human mobility data learns embeddings capable, through fine-tuning,
of developing a deep understanding of the target geography and its
corresponding mobility patterns. Utilizing an adaptation framework, we evaluate
the performance of our pre-trained embeddings in encapsulating a broad spectrum
of concepts directly and indirectly related to human mobility. This includes
basic notions, such as geographic location and distance, and extends to more
complex constructs, such as administrative divisions and land cover. Our
extensive empirical analysis reveals a substantial performance boost gained
from pre-training, reaching up to 38% in tasks such as tree-cover regression.
We attribute this result to the ability of the pre-training to uncover
meaningful patterns hidden in the raw data, beneficial for modeling relevant
high-level concepts. The pre-trained embeddings emerge as robust
representations of regions and trajectories, potentially valuable for a wide
range of downstream applications.

基于国家级无标签人类流动数据进行事先训练的转换器能够通过微调形成对目标地理区域及其相应流动模式的深入理解，我们的预先训练嵌入在涵盖与人类流动直接和间接相关的广泛概念方面表现出良好性能，包括地理位置、距离、行政区划和土地覆盖等，通过大量实证分析，我们发现预先训练可以显著提升性能，在树木覆盖回归等任务中达到 38％，我们将这一结果归因于预先训练能够揭示原始数据中隐藏的有意义模式，并有助于建模相关的高级概念，预先训练的嵌入成为表征区域和轨迹的强大工具，在广泛的后续应用中具有潜在价值。