Privacy concerns have attracted increasing attention in data-driven products
and services. Existing legislation forbids arbitrary processing of personal
data collected from individuals. Generating synthetic versions of such data
with a formal privacy guarantee such as differential privacy (DP) is considered
to be a solution to address privacy concerns. In this direction, we show a
simple, practical, and effective recipe in the text domain: simply fine-tuning
a generative language model with DP allows us to generate useful synthetic text
while mitigating privacy concerns. Through extensive empirical analyses, we
demonstrate that our method produces synthetic data that is competitive in
terms of utility with its non-private counterpart and meanwhile provides strong
protection against potential privacy leakages.

本文讨论隐私问题在数据驱动产品和服务中引起的关注，提出了一种利用差分隐私的生成式语言模型的简单、实用和有效方法，以生成有用的合成文本同时缓解隐私问题。经过广泛的实证分析，我们证明了我们的方法产生的合成数据在实用性上与其非私有对应物相当竞争，同时提供了强大的保护措施以防止潜在的隐私泄漏。