Heterogeneous graph neural networks (HGNNs) have been blossoming in recent
years, but the unique data processing and evaluation setups used by each work
obstruct a full understanding of their advancements. In this work, we present a
systematical reproduction of 12 recent HGNNs by using their official codes,
datasets, settings, and hyperparameters, revealing surprising findings about
the progress of HGNNs. We find that the simple homogeneous GNNs, e.g., GCN and
GAT, are largely underestimated due to improper settings. GAT with proper
inputs can generally match or outperform all existing HGNNs across various
scenarios. To facilitate robust and reproducible HGNN research, we construct
the Heterogeneous Graph Benchmark (HGB), consisting of 11 diverse datasets with
three tasks. HGB standardizes the process of heterogeneous graph data splits,
feature processing, and performance evaluation. Finally, we introduce a simple
but very strong baseline Simple-HGN--which significantly outperforms all
previous models on HGB--to accelerate the advancement of HGNNs in the future.

本文通过使用官方代码、数据集、设置和超参数，对 12 个最近的异质图神经网络进行了系统的复现，揭示了关于 HGNNs 进展的惊人发现。发现由于不当的设置，简单的同质 GNNs 被大大低估，GAT 在适当的输入情况下可以在各种情况下与或胜过所有现有的 HGNNs。为了促进鲁棒且可重复的 HGNN 研究，我们构建了异质图基准 (HGB)，其中包括具有三个任务的 11 个不同的数据集。HGB 规范了异质图数据分割、特征处理和性能评估的流程。最后，我们介绍了一个简单但非常强大的基准 Simple-HGN，它在 HGB 上明显优于所有先前的模型，以加速未来 HGNNs 的发展。