BriefGPT.xyz
Sep, 2020
差分隐私语言模型受益于公共预训练
Differentially Private Language Models Benefit from Public Pre-training
HTML
PDF
Gavin Kerrigan, Dylan Slack, Jens Tuyls
TL;DR
通过微调基于公共语料库的模型来实现高质量和隐私保护的语言模型，提高私有领域的模型性能，让其成为可能。
Abstract
language modeling
is a keystone task in natural language processing. When training a language model on sensitive information,
differential privacy
(DP) allows us to quantify the degree to which our private data i
→