Topic models have been prevalent for decades with various applications.
However, existing topic models commonly suffer from the notorious topic
collapsing: discovered topics semantically collapse towards each other, leading
to highly repetitive topics, insufficient topic discovery, and damaged model
interpretability. In this paper, we propose a new neural topic model, Embedding
Clustering Regularization Topic Model (ECRTM). Besides the existing
reconstruction error, we propose a novel Embedding Clustering Regularization
(ECR), which forces each topic embedding to be the center of a separately
aggregated word embedding cluster in the semantic space. This enables each
produced topic to contain distinct word semantics, which alleviates topic
collapsing. Regularized by ECR, our ECRTM generates diverse and coherent topics
together with high-quality topic distributions of documents. Extensive
experiments on benchmark datasets demonstrate that ECRTM effectively addresses
the topic collapsing issue and consistently surpasses state-of-the-art
baselines in terms of topic quality, topic distributions of documents, and
downstream classification tasks.

本文提出了一种新的神经主题模型 —— 嵌入聚类正则化主题模型 (ECRTM)，通过嵌入聚类正则化 (ER) 使每个产生的主题包含不同的词义，从而缓解了主题崩溃问题，并在主题质量、文档主题分布和下游分类任务等方面持续超越了最先进的基线。