Text-based delivery addresses, as the data foundation for logistics systems,
contain abundant and crucial location information. How to effectively encode
the delivery address is a core task to boost the performance of downstream
tasks in the logistics system. Pre-trained Models (PTMs) designed for Natural
Language Process (NLP) have emerged as the dominant tools for encoding semantic
information in text. Though promising, those NLP-based PTMs fall short of
encoding geographic knowledge in the delivery address, which considerably trims
down the performance of delivery-related tasks in logistic systems such as
Cainiao. To tackle the above problem, we propose a domain-specific pre-trained
model, named G2PTL, a Geography-Graph Pre-trained model for delivery address in
Logistics field. G2PTL combines the semantic learning capabilities of text
pre-training with the geographical-relationship encoding abilities of graph
modeling. Specifically, we first utilize real-world logistics delivery data to
construct a large-scale heterogeneous graph of delivery addresses, which
contains abundant geographic knowledge and delivery information. Then, G2PTL is
pre-trained with subgraphs sampled from the heterogeneous graph. Comprehensive
experiments are conducted to demonstrate the effectiveness of G2PTL through
four downstream tasks in logistics systems on real-world datasets. G2PTL has
been deployed in production in Cainiao's logistics system, which significantly
improves the performance of delivery-related tasks.

提出了一种新型的基于地理关系的预训练模型 G2PTL，它采用了真实的物流交付数据构建了一个包含丰富地理知识和交付信息的大规模异构图，并通过采样子图进行预训练，通过真实世界数据的四项下游任务的全面实验表明，G2PTL 能够显著提高物流系统中与交付相关任务的性能。