Recently there has been a surge of interest in extending the success of large
language models (LLMs) to graph modality, such as social networks and
molecules. As LLMs are predominantly trained with 1D text data, most existing
approaches adopt a graph neural network to represent a graph as a series of
node tokens and feed these tokens to LLMs for graph-language alignment. Despite
achieving some successes, existing approaches have overlooked the hierarchical
structures that are inherent in graph data. Especially, in molecular graphs,
the high-order structural information contains rich semantics of molecular
functional groups, which encode crucial biochemical functionalities of the
molecules. We establish a simple benchmark showing that neglecting the
hierarchical information in graph tokenization will lead to subpar
graph-language alignment and severe hallucination in generated outputs. To
address this problem, we propose a novel strategy called HIerarchical GrapH
Tokenization (HIGHT). HIGHT employs a hierarchical graph tokenizer that
extracts and encodes the hierarchy of node, motif, and graph levels of
informative tokens to improve the graph perception of LLMs. HIGHT also adopts
an augmented graph-language supervised fine-tuning dataset, enriched with the
hierarchical graph information, to further enhance the graph-language
alignment. Extensive experiments on 7 molecule-centric benchmarks confirm the
effectiveness of HIGHT in reducing hallucination by 40%, as well as significant
improvements in various molecule-language downstream tasks.

利用分层图标记的新策略（HIGHT）在分子图中增强了大型语言模型（LLMs）的图 - 语言对齐性和生成输出的准确性。