Topic models are one of the compelling methods for discovering latent
semantics in a document collection. However, it assumes that a document has
sufficient co-occurrence information to be effective. However, in short texts,
co-occurrence information is minimal, which results in feature sparsity in
document representation. Therefore, existing topic models (probabilistic or
neural) mostly fail to mine patterns from them to generate coherent topics. In
this paper, we take a new approach to short-text topic modeling to address the
data-sparsity issue by extending short text into longer sequences using
existing pre-trained language models (PLMs). Besides, we provide a simple
solution extending a neural topic model to reduce the effect of noisy
out-of-topics text generation from PLMs. We observe that our model can
substantially improve the performance of short-text topic modeling. Extensive
experiments on multiple real-world datasets under extreme data sparsity
scenarios show that our models can generate high-quality topics outperforming
state-of-the-art models.

在本论文中，我们采取了一种新的方法来解决短文本主题建模中的数据稀疏问题，通过利用现有的预训练语言模型将短文本扩展为更长的序列。此外，我们提供了一种简单的解决方案，通过扩展神经主题模型来减少预训练语言模型生成的与主题无关的噪声文本的影响。我们观察到我们的模型能够显著改善短文本主题建模的性能。在极度数据稀疏的情况下，对多个真实数据集进行的大量实验证明了我们的模型可以生成优质主题，胜过现有最先进的模型。