Recent research shows that large language models are susceptible to privacy
attacks that infer aspects of the training data. However, it is unclear if
simpler generative models, like topic models, share similar vulnerabilities. In
this work, we propose an attack against topic models that can confidently
identify members of the training data in Latent Dirichlet Allocation. Our
results suggest that the privacy risks associated with generative modeling are
not restricted to large neural models. Additionally, to mitigate these
vulnerabilities, we explore differentially private (DP) topic modeling. We
propose a framework for private topic modeling that incorporates DP vocabulary
selection as a pre-processing step, and show that it improves privacy while
having limited effects on practical utility.

最近的研究表明，大型语言模型容易受到侵犯隐私的攻击，推断出训练数据的某些方面。然而，目前尚不清楚是否简单的生成模型（如主题模型）具有类似的弱点。本文提出了一种针对主题模型的攻击方法，可以自信地识别潜在狄利克雷分布中的训练数据成员。我们的结果表明，与生成神经模型相关的隐私风险并不仅限于大型模型。此外，为了减轻这些弱点，我们探讨了差分隐私的主题模型。我们提出了一个框架，将差分隐私词汇选择作为预处理步骤纳入私有主题模型，并展示它在保护隐私方面的改进，对实际效用影响较小。