We present Bi-Level Attention-Based Relational Graph Convolutional Networks
(BR-GCN), unique neural network architectures that utilize masked
self-attentional layers with relational graph convolutions, to effectively
operate on highly multi-relational data. BR-GCN models use bi-level attention
to learn node embeddings through (1) node-level attention, and (2)
relation-level attention. The node-level self-attentional layers use
intra-relational graph interactions to learn relation-specific node embeddings
using a weighted aggregation of neighborhood features in a sparse subgraph
region. The relation-level self-attentional layers use inter-relational graph
interactions to learn the final node embeddings using a weighted aggregation of
relation-specific node embeddings. The BR-GCN bi-level attention mechanism
extends Transformer-based multiplicative attention from the natural language
processing (NLP) domain, and Graph Attention Networks (GAT)-based attention, to
large-scale heterogeneous graphs (HGs). On node classification, BR-GCN
outperforms baselines from 0.29% to 14.95% as a stand-alone model, and on link
prediction, BR-GCN outperforms baselines from 0.02% to 7.40% as an auto-encoder
model. We also conduct ablation studies to evaluate the quality of BR-GCN's
relation-level attention and discuss how its learning of graph structure may be
transferred to enrich other graph neural networks (GNNs). Through various
experiments, we show that BR-GCN's attention mechanism is both scalable and
more effective in learning compared to state-of-the-art GNNs.

我们提出了 Bi-Level Attention-Based Relational Graph Convolutional Networks (BR-GCN)，这是一种利用具有关系图卷积的掩码自注意力层的独特神经网络架构，能够有效处理高度多关系数据。通过节点级别和关系级别的注意力机制，BR-GCN 模型通过节点级自注意层和关系级自注意层学习节点嵌入，从而在大规模异构图上超过了现有的基线模型。

多关系图的分层注意力模型

Hierarchical Attention Models for Multi-Relational Graphs

Graph neural network, as a powerful graph representation technique based on
deep learning, has shown superior performance and attracted considerable
research interest. However, it has not been fully considered in graph neural
network for heterogeneous graph which contains different types of nodes and
links. The heterogeneity and rich semantic information bring great challenges
for designing a graph neural network for heterogeneous graph. Recently, one of
the most exciting advancements in deep learning is the attention mechanism,
whose great potential has been well demonstrated in various areas. In this
paper, we first propose a novel heterogeneous graph neural network based on the
hierarchical attention, including node-level and semantic-level attentions.
Specifically, the node-level attention aims to learn the importance between a
node and its metapath based neighbors, while the semantic-level attention is
able to learn the importance of different meta-paths. With the learned
importance from both node-level and semantic-level attention, the importance of
node and meta-path can be fully considered. Then the proposed model can
generate node embedding by aggregating features from meta-path based neighbors
in a hierarchical manner. Extensive experimental results on three real-world
heterogeneous graphs not only show the superior performance of our proposed
model over the state-of-the-arts, but also demonstrate its potentially good
interpretability for graph analysis.

该论文提出了一种基于分层注意力机制的异构图神经网络，包括节点层次和语义层次的注意力机制，通过从元路径邻居中汇总特征来生成节点嵌入，其在三个真实的异构图上表现出优越性能且有助于图分析。