While graph neural networks (GNNs) are widely used for node and graph
representation learning tasks, the reliability of GNN uncertainty estimates
under distribution shifts remains relatively under-explored. Indeed, while
post-hoc calibration strategies can be used to improve in-distribution
calibration, they need not also improve calibration under distribution shift.
However, techniques which produce GNNs with better intrinsic uncertainty
estimates are particularly valuable, as they can always be combined with
post-hoc strategies later. Therefore, in this work, we propose G-$\Delta$UQ, a
novel training framework designed to improve intrinsic GNN uncertainty
estimates. Our framework adapts the principle of stochastic data centering to
graph data through novel graph anchoring strategies, and is able to support
partially stochastic GNNs. While, the prevalent wisdom is that fully stochastic
networks are necessary to obtain reliable estimates, we find that the
functional diversity induced by our anchoring strategies when sampling
hypotheses renders this unnecessary and allows us to support G-$\Delta$UQ on
pretrained models. Indeed, through extensive evaluation under covariate,
concept and graph size shifts, we show that G-$\Delta$UQ leads to better
calibrated GNNs for node and graph classification. Further, it also improves
performance on the uncertainty-based tasks of out-of-distribution detection and
generalization gap estimation. Overall, our work provides insights into
uncertainty estimation for GNNs, and demonstrates the utility of G-$\Delta$UQ
in obtaining reliable estimates.

我们提出了 G-DeltaUQ，这是一个新的训练框架，旨在改进图神经网络的内在不确定性估计，通过独特的图锚定策略适应图数据，从而在节点和图分类中实现更好的标定准确度。

用于图神经网络的准确可扩展的认知不确定性估计

Accurate and Scalable Estimation of Epistemic Uncertainty for Graph  Neural Networks

Although Transformer has achieved great success in natural language process
and computer vision, it has difficulty generalizing to medium and large-scale
graph data for two important reasons: (i) High complexity. (ii) Failing to
capture the complex and entangled structure information. In graph
representation learning, Graph Neural Networks(GNNs) can fuse the graph
structure and node attributes but have limited receptive fields. Therefore, we
question whether can we combine Transformers and GNNs to help each other. In
this paper, we propose a new model named TransGNN where the Transformer layer
and GNN layer are used alternately to improve each other. Specifically, to
expand the receptive field and disentangle the information aggregation from
edges, we propose using Transformer to aggregate more relevant nodes'
information to improve the message passing of GNNs. Besides, to capture the
graph structure information, we utilize positional encoding and make use of the
GNN layer to fuse the structure into node attributes, which improves the
Transformer in graph data. We also propose to sample the most relevant nodes
for Transformer and two efficient samples update strategies to lower the
complexity. At last, we theoretically prove that TransGNN is more expressive
than GNNs only with extra linear complexity. The experiments on eight datasets
corroborate the effectiveness of TransGNN on node and graph classification
tasks.

在这篇论文中，我们提出了一种名为 TransGNN 的新模型，其中 Transformer 层和 GNN 层交替使用以相互改进。我们利用 Transformer 来聚合更相关的节点信息以改善 GNNs 的消息传递，并利用位置编码和 GNN 层将图结构融入节点属性中，改善 Transformer 在图数据中的表现。通过理论证明，我们证明了 TransGNN 相对于仅具有额外线性复杂度的 GNNs 更具表达力。在八个数据集上的实验证实了 TransGNN 在节点和图分类任务上的有效性。

Transformer 和 GNN 能互助吗？

Can Transformer and GNN Help Each Other?

Implicit graph neural networks (GNNs) have emerged as a potential approach to
enable GNNs to capture long-range dependencies effectively. However, poorly
designed implicit GNN layers can experience over-smoothing or may have limited
adaptability to learn data geometry, potentially hindering their performance in
graph learning problems. To address these issues, we introduce a geometric
framework to design implicit graph diffusion layers based on a parameterized
graph Laplacian operator. Our framework allows learning the geometry of vertex
and edge spaces, as well as the graph gradient operator from data. We further
show how implicit GNN layers can be viewed as the fixed-point solution of a
Dirichlet energy minimization problem and give conditions under which it may
suffer from over-smoothing. To overcome the over-smoothing problem, we design
our implicit graph diffusion layer as the solution of a Dirichlet energy
minimization problem with constraints on vertex features, enabling it to trade
off smoothing with the preservation of node feature information. With an
appropriate hyperparameter set to be larger than the largest eigenvalue of the
parameterized graph Laplacian, our framework guarantees a unique equilibrium
and quick convergence. Our models demonstrate better performance than leading
implicit and explicit GNNs on benchmark datasets for node and graph
classification tasks, with substantial accuracy improvements observed for some
datasets.

设计基于参数化图拉普拉斯算子的隐式图扩散层，从而在节点和图分类任务中展现了强化的性能，通过减少超平滑问题，权衡平滑和节点特征信息的保持。