BriefGPT.xyz
Mar, 2024
通过合成文本生成实现差分隐私知识蒸馏
Differentially Private Knowledge Distillation via Synthetic Text Generation
HTML
PDF
James Flemings, Murali Annavaram
TL;DR
利用差分隐私的知识蒸馏算法,通过利用合成数据和教师模型的输出分布,成功地在压缩自回归型大型语言模型的同时保护训练数据的隐私。
Abstract
large language models
(LLMs) are achieving state-of-the-art performance in many different downstream tasks. However, the increasing urgency of data privacy requires LLMs to train with
differential privacy
(DP) on
→