Text-to-image diffusion models have been shown to suffer from sample-level
memorization, possibly reproducing near-perfect replica of images that they are
trained on, which may be undesirable. To remedy this issue, we develop the
first differentially private (DP) retrieval-augmented generation algorithm that
is capable of generating high-quality image samples while providing provable
privacy guarantees. Specifically, we assume access to a text-to-image diffusion
model trained on a small amount of public data, and design a DP retrieval
mechanism to augment the text prompt with samples retrieved from a private
retrieval dataset. Our \emph{differentially private retrieval-augmented
diffusion model} (DP-RDM) requires no fine-tuning on the retrieval dataset to
adapt to another domain, and can use state-of-the-art generative models to
generate high-quality image samples while satisfying rigorous DP guarantees.
For instance, when evaluated on MS-COCO, our DP-RDM can generate samples with a
privacy budget of $\epsilon=10$, while providing a $3.5$ point improvement in
FID compared to public-only retrieval for up to $10,000$ queries.

发展首个具有可证明的隐私保证并能生成高质量图像样本的差分隐私检索增强生成算法，通过在文本提示中引入从私有检索数据集检索的样本，无需在检索数据集上微调，利用先进的生成模型生成高质量图像样本，并提供隐私保证。