We present Bi-Level Attention-Based Relational Graph Convolutional Networks
(BR-GCN), unique neural network architectures that utilize masked
self-attentional layers with relational graph convolutions, to effectively
operate on highly multi-relational data. BR-GCN models use bi-level attention
to learn node embeddings through (1) node-level attention, and (2)
relation-level attention. The node-level self-attentional layers use
intra-relational graph interactions to learn relation-specific node embeddings
using a weighted aggregation of neighborhood features in a sparse subgraph
region. The relation-level self-attentional layers use inter-relational graph
interactions to learn the final node embeddings using a weighted aggregation of
relation-specific node embeddings. The BR-GCN bi-level attention mechanism
extends Transformer-based multiplicative attention from the natural language
processing (NLP) domain, and Graph Attention Networks (GAT)-based attention, to
large-scale heterogeneous graphs (HGs). On node classification, BR-GCN
outperforms baselines from 0.29% to 14.95% as a stand-alone model, and on link
prediction, BR-GCN outperforms baselines from 0.02% to 7.40% as an auto-encoder
model. We also conduct ablation studies to evaluate the quality of BR-GCN's
relation-level attention and discuss how its learning of graph structure may be
transferred to enrich other graph neural networks (GNNs). Through various
experiments, we show that BR-GCN's attention mechanism is both scalable and
more effective in learning compared to state-of-the-art GNNs.

我们提出了 Bi-Level Attention-Based Relational Graph Convolutional Networks (BR-GCN)，这是一种利用具有关系图卷积的掩码自注意力层的独特神经网络架构，能够有效处理高度多关系数据。通过节点级别和关系级别的注意力机制，BR-GCN 模型通过节点级自注意层和关系级自注意层学习节点嵌入，从而在大规模异构图上超过了现有的基线模型。