LLM-powered chatbots are becoming widely adopted in applications such as healthcare, personal assistants, industry hiring decisions, etc. In many of these cases, chatbots are fed sensitive, personal information in their prompts, as samples for in-context learning, retrieved records from a database, or as part of the conversation. The information provided in the prompt could directly appear in the output, which might have privacy ramifications if there is sensitive information there. As such, in this paper, we aim to understand the input copying and regurgitation capabilities of these models during inference and how they can be directly instructed to limit this copying by complying with regulations such as HIPAA and GDPR, based on their internal knowledge of them. More specifically, we find that when ChatGPT is prompted to summarize cover letters of a 100 candidates, it would retain personally identifiable information (PII) verbatim in 57.4% of cases, and we find this retention to be non-uniform between different subgroups of people, based on attributes such as gender identity. We then probe ChatGPT's perception of privacy-related policies and privatization mechanisms by directly instructing it to provide compliant outputs and observe a significant omission of PII from output.

本文旨在探究聊天机器人的输入复制和反刍能力，并通过直接指示来遵守HIPAA和GDPR等规定，限制复制，以减少隐私风险。通过概括100名应聘者的求职信，我们发现ChatGPT在57.4％的情况下会保留PII，并发现在不同的人员子群组之间存在非统一的差异，然而，通过指示，我们也发现ChatGPT在输出中能够显著省略PII。

聊天机器人是否准备好应对隐私敏感的应用？输入翻转和提示清除的调查