In text classification tasks, fine tuning pretrained language models like
BERT and GPT-3 yields competitive accuracy; however, both methods require
pretraining on large text datasets. In contrast, general topic modeling methods
possess the advantage of analyzing documents to extract meaningful patterns of
words without the need of pretraining. To leverage topic modeling's
unsupervised insights extraction on text classification tasks, we develop the
Knowledge Distillation Semi-supervised Topic Modeling (KDSTM). KDSTM requires
no pretrained embeddings, few labeled documents and is efficient to train,
making it ideal under resource constrained settings. Across a variety of
datasets, our method outperforms existing supervised topic modeling methods in
classification accuracy, robustness and efficiency and achieves similar
performance compare to state of the art weakly supervised text classification
methods.

本文提出了一种名为知识蒸馏半监督主题模型（KDSTM）的方法，该方法可以利用主题模型的无监督特征提取来完成文本分类任务，而无需预先训练。该方法具有较高的效率和准确性，相对于基于监督的分类模型更具鲁棒性。