BriefGPT.xyz
Nov, 2023
大规模嵌入模型的稀疏保持差分私有训练
Sparsity-Preserving Differentially Private Training of Large Embedding Models
HTML
PDF
Badih Ghazi, Yangsibo Huang, Pritish Kamath, Ravi Kumar, Pasin Manurangsi...
TL;DR
使用DP-SGD算法对大型嵌入模型进行隐私训练时，为了维持梯度稀疏性，我们提出了两个新算法DP-FEST和DP-AdaFEST，能够在保持相当准确性的同时，实现梯度大小的大幅度降低（$10^6 imes$）。
Abstract
As the use of
large embedding models
in recommendation systems and language applications increases, concerns over
user data privacy
have also risen.
→