This paper examines the challenging problem of learning representations of entities and relations in a complex multi-relational knowledge graph. We propose HittER, a Hierarchical Transformer model to jointly learn Entity-relation composition and Relational contextualization based on a source entity's neighborhood. Our proposed model consists of two different Transformer blocks: the bottom block extracts features of each entity-relation pair in the local neighborhood of the source entity and the top block aggregates the relational information from the outputs of the bottom block. We further design a masked entity prediction task to balance information from the relational context and the source entity itself. Evaluated on the task of link prediction, our approach achieves new state-of-the-art results on two standard benchmark datasets FB15K-237 and WN18RR.

本文针对多关系知识图谱中实体和关系的学习表示问题，提出了一种基于分层Transformer模型的方法，该方法可以联合学习实体关系组合和基于源实体社区的关系上下文。实验结果表明HittER模型在多个链接预测数据集上取得了最新的最优结果。此外，我们还提出了一种简单的方法将HittER集成到BERT，并在两个Freebase问答数据集上证明了其有效性。

HittER: 分层Transformer用于知识图谱嵌入