Correlated topic modeling has been limited to small model and problem sizes due to their high computational cost and poor scaling. In this paper, we propose a new model which learns compact topic embeddings and captures topic correlations through the closeness between the topic vectors. Our method enables efficient inference in the low-dimensional embedding space, reducing previous cubic or quadratic time complexity to linear w.r.t the topic size. We further speedup variational inference with a fast sampler to exploit sparsity of topic occurrence. Extensive experiments show that our approach is capable of handling model and data scales which are several orders of magnitude larger than existing correlation results, without sacrificing modeling quality by providing competitive or superior performance in document classification and retrieval.

本文提出了一种新的模型，通过主题向量之间的接近程度来学习紧凑的主题嵌入，并捕捉主题相关性，从而降低了以前的三次或二次时间复杂度至线性，同时利用快速采样器加速变分推断以利用主题出现的稀疏性，在不牺牲建模质量的前提下，能够处理比现有相关结果大几个数量级的模型和数据规模，并在文档分类和检索中提供竞争性或优越的性能。

主题嵌入式高效相关主题建模