We consider the problem of language model inversion: given outputs of a
language model, we seek to extract the prompt that generated these outputs. We
develop a new black-box method, output2prompt, that learns to extract prompts
without access to the model's logits and without adversarial or jailbreaking
queries. In contrast to previous work, output2prompt only needs outputs of
normal user queries. To improve memory efficiency, output2prompt employs a new
sparse encoding techique. We measure the efficacy of output2prompt on a variety
of user and system prompts and demonstrate zero-shot transferability across
different LLMs.

给定一个语言模型输出，我们考虑语言模型反演的问题：我们寻找生成这些输出的提示信息。我们开发了一种新的黑盒方法 output2prompt，它能够学习在不访问模型的 logits 或使用对抗性或越狱查询的情况下提取提示信息。与以前的工作不同，output2prompt 只需要普通用户查询的输出。为了提高内存效率，output2prompt 采用一种新的稀疏编码技术。我们衡量了 output2prompt 在各种用户和系统提示上的有效性，并展示了在不同语言模型之间的零 - shot 可迁移性。

通过反向获取 LLM 输出来提取提示

Extracting Prompts by Inverting LLM Outputs

Large Language Models (LLMs) like ChatGPT face `jailbreak' challenges, where
safeguards are bypassed to produce ethically harmful prompts. This study
introduces a simple black-box method to effectively generate jailbreak prompts,
overcoming the limitations of high complexity and computational costs
associated with existing methods. The proposed technique iteratively rewrites
harmful prompts into non-harmful expressions using the target LLM itself, based
on the hypothesis that LLMs can directly sample safeguard-bypassing
expressions. Demonstrated through experiments with ChatGPT (GPT-3.5 and GPT-4)
and Gemini-Pro, this method achieved an attack success rate of over 80% within
an average of 5 iterations and remained effective despite model updates. The
jailbreak prompts generated were naturally-worded and concise, suggesting they
are less detectable. The results indicate that creating effective jailbreak
prompts is simpler than previously considered, and black-box jailbreak attacks
pose a more serious security threat.

通过使用以 ChatGPT 为目标的简单黑盒方法，本研究有效地生成越过伦理规定的提示，突破了现有方法的复杂性和计算成本的限制，该方法通过 LLM 自身将有害的提示迭代地重写为无害表达式，该研究结果表明，创建有效的越狱提示比以前认为的更简单，并且黑盒越狱攻击构成了更严重的安全威胁。