Existing NTMs with contrastive learning suffer from the sample bias problem
owing to the word frequency-based sampling strategy, which may result in false
negative samples with similar semantics to the prototypes. In this paper, we
aim to explore the efficient sampling strategy and contrastive learning in NTMs
to address the aforementioned issue. We propose a new sampling assumption that
negative samples should contain words that are semantically irrelevant to the
prototype. Based on it, we propose the graph contrastive topic model (GCTM),
which conducts graph contrastive learning (GCL) using informative positive and
negative samples that are generated by the graph-based sampling strategy
leveraging in-depth correlation and irrelevance among documents and words. In
GCTM, we first model the input document as the document word bipartite graph
(DWBG), and construct positive and negative word co-occurrence graphs (WCGs),
encoded by graph neural networks, to express in-depth semantic correlation and
irrelevance among words. Based on the DWBG and WCGs, we design the
document-word information propagation (DWIP) process to perform the edge
perturbation of DWBG, based on multi-hop correlations/irrelevance among
documents and words. This yields the desired negative and positive samples,
which will be utilized for GCL together with the prototypes to improve learning
document topic representations and latent topics. We further show that GCL can
be interpreted as the structured variational graph auto-encoder which maximizes
the mutual information of latent topic representations of different
perspectives on DWBG. Experiments on several benchmark datasets demonstrate the
effectiveness of our method for topic coherence and document representation
learning compared with existing SOTA methods.

本文提出了一种基于图的对比学习方法，使用生成的正负样本和原型来改善学习文档主题表示和潜在主题。通过对几个基准数据集的实验，证明了该方法在主题连贯性和文档表示学习方面的有效性。

图形对比主题模型

Graph Contrastive Topic Model

Recently, neural topic models (NTMs) have been incorporated into pre-trained
language models (PLMs), to capture the global semantic information for text
summarization. However, in these methods, there remain limitations in the way
they capture and integrate the global semantic information. In this paper, we
propose a novel model, the graph contrastive topic enhanced language model
(GRETEL), that incorporates the graph contrastive topic model with the
pre-trained language model, to fully leverage both the global and local
contextual semantics for long document extractive summarization. To better
capture and incorporate the global semantic information into PLMs, the graph
contrastive topic model integrates the hierarchical transformer encoder and the
graph contrastive learning to fuse the semantic information from the global
document context and the gold summary. To this end, GRETEL encourages the model
to efficiently extract salient sentences that are topically related to the gold
summary, rather than redundant sentences that cover sub-optimal topics.
Experimental results on both general domain and biomedical datasets demonstrate
that our proposed method outperforms SOTA methods.

本文提出了图对比主题增强语言模型 (GRETEL)，该模型将图对比主题模型与预训练语言模型相结合，以充分利用全局和局部语境语义进行长文本提取式摘要。实验结果表明，该方法优于目前最佳方法。