It has been demonstrated that hidden representation learned by a deep model can encode private information of the input, hence can be exploited to recover such information with reasonable accuracy. To address this issue, we propose a novel approach called Differentially Private Neural Representation (DPNR) to preserve the privacy of the extracted representation from text. DPNR utilises Differential Privacy (DP) to provide a formal privacy guarantee. Further, we show that masking words via dropout can further enhance privacy. To maintain utility of the learned representation, we integrate DP-noisy representation into a robust training process to derive a robust target model, which also helps for model fairness over various demographic variables. Experimental results on benchmark datasets under various parameter settings demonstrate that DPNR largely reduces privacy leakage without significantly sacrificing the main task performance.

提出了一种名为DPNR的深度学习方法，该方法使用差分隐私实现了对文本中提取的表示的隐私保护，并通过在强健训练中集成DP噪声表示来维护学习表示的效用。实验结果表明，DPNR可以在不显著牺牲主要任务性能的情况下大大减少隐私泄露。

自然语言处理的差分隐私表示：形式保证和隐私公平性的实证研究