Hypergraphs, with their capacity to depict high-order relationships, have
emerged as a significant extension of traditional graphs. Although Graph Neural
Networks (GNNs) have remarkable performance in graph representation learning,
their extension to hypergraphs encounters challenges due to their intricate
structures. Furthermore, current hypergraph transformers, a special variant of
GNN, utilize semantic feature-based self-attention, ignoring topological
attributes of nodes and hyperedges. To address these challenges, we propose a
Topology-guided Hypergraph Transformer Network (THTN). In this model, we first
formulate a hypergraph from a graph while retaining its structural essence to
learn higher-order relations within the graph. Then, we design a simple yet
effective structural and spatial encoding module to incorporate the topological
and spatial information of the nodes into their representation. Further, we
present a structure-aware self-attention mechanism that discovers the important
nodes and hyperedges from both semantic and structural viewpoints. By
leveraging these two modules, THTN crafts an improved node representation,
capturing both local and global topological expressions. Extensive experiments
conducted on node classification tasks demonstrate that the performance of the
proposed model consistently exceeds that of the existing approaches.

提出了一种拓扑引导的超图变换网络，通过保留结构本质学习图中的高阶关系，设计了简洁而有效的结构和空间编码模块将节点的拓扑和空间信息融入其表示，并提出了一种结构感知的自注意机制，从语义和结构角度发现重要节点和超边。实验证明，所提模型在节点分类任务上的性能始终优于现有方法。

拓扑指导的超图变换网络：揭示结构洞察以改进表示

Topology-guided Hypergraph Transformer Network: Unveiling Structural  Insights for Improved Representation

Symbolic knowledge graphs (KGs) play a pivotal role in knowledge-centric
applications such as search, question answering and recommendation. As
contemporary language models (LMs) trained on extensive textual data have
gained prominence, researchers have extensively explored whether the parametric
knowledge within these models can match up to that present in knowledge graphs.
Various methodologies have indicated that enhancing the size of the model or
the volume of training data enhances its capacity to retrieve symbolic
knowledge, often with minimal or no human supervision. Despite these
advancements, there is a void in comprehensively evaluating whether LMs can
encompass the intricate topological and semantic attributes of KGs, attributes
crucial for reasoning processes. In this work, we provide an exhaustive
evaluation of language models of varying sizes and capabilities. We construct
nine qualitative benchmarks that encompass a spectrum of attributes including
symmetry, asymmetry, hierarchy, bidirectionality, compositionality, paths,
entity-centricity, bias and ambiguity. Additionally, we propose novel
evaluation metrics tailored for each of these attributes. Our extensive
evaluation of various LMs shows that while these models exhibit considerable
potential in recalling factual information, their ability to capture intricate
topological and semantic traits of KGs remains significantly constrained. We
note that our proposed evaluation metrics are more reliable in evaluating these
abilities than the existing metrics. Lastly, some of our benchmarks challenge
the common notion that larger LMs (e.g., GPT-4) universally outshine their
smaller counterparts (e.g., BERT).

基于符号的知识图谱、语言模型和评估指标对知识图谱的拓扑属性和语义属性进行了详尽的评估，发现语言模型在提取符号知识方面能力受到限制。