Hypergraphs are marked by complex topology, expressing higher-order
interactions among multiple entities with hyperedges. Lately, hypergraph-based
deep learning methods to learn informative data representations for the problem
of node classification on text-attributed hypergraphs have garnered increasing
research attention. However, existing methods struggle to simultaneously
capture the full extent of hypergraph structural information and the rich
linguistic attributes inherent in the nodes attributes, which largely hampers
their effectiveness and generalizability. To overcome these challenges, we
explore ways to further augment a pretrained BERT model with specialized
hypergraph-aware layers for the task of node classification. Such layers
introduce higher-order structural inductive bias into the language model, thus
improving the model's capacity to harness both higher-order context information
from the hypergraph structure and semantic information present in text. In this
paper, we propose a new architecture, HyperBERT, a mixed text-hypergraph model
which simultaneously models hypergraph relational structure while maintaining
the high-quality text encoding capabilities of a pre-trained BERT. Notably,
HyperBERT presents results that achieve a new state-of-the-art on 5 challenging
text-attributed hypergraph node classification benchmarks.

开发一种新的架构 HyperBERT，结合了超图的关系结构和预训练 BERT 模型，用于解决节点分类问题，并在 5 个具有挑战性的文本属性超图节点分类基准测试中取得了最新的最佳结果。