Pre-trained language models have led to a new state-of-the-art in many NLP
tasks. However, for topic modeling, statistical generative models such as LDA
are still prevalent, which do not easily allow incorporating contextual word
vectors. They might yield topics that do not align very well with human
judgment. In this work, we propose a novel topic modeling and inference
algorithm. We suggest a bag of sentences (BoS) approach using sentences as the
unit of analysis. We leverage pre-trained sentence embeddings by combining
generative process models with clustering. We derive a fast inference algorithm
based on expectation maximization, hard assignments, and an annealing process.
Our evaluation shows that our method yields state-of-the art results with
relatively little computational demands. Our methods is more flexible compared
to prior works leveraging word embeddings, since it provides the possibility to
customize topic-document distributions using priors. Code is at
https://github.com/JohnTailor/BertSenClu.

本文提出了一种基于句子嵌入和生成过程模型相结合的话题建模和推断算法，使用期望最大化、硬分配和退火过程推导出快速推理算法。在评估中，我们的方法以相对较少的计算要求取得了最先进的结果。