Large Language Model (LLM) alignment aims to ensure that LLM outputs match
with human values. Researchers have demonstrated the severity of alignment
problems with a large spectrum of jailbreak techniques that can induce LLMs to
produce malicious content during conversations. Finding the corresponding
jailbreaking prompts usually requires substantial human intelligence or
computation resources. In this paper, we report that LLMs have different levels
of alignment in various contexts. As such, by systematically constructing many
contexts, called worlds, leveraging a Domain Specific Language describing
possible worlds (e.g., time, location, characters, actions and languages) and
the corresponding compiler, we can cost-effectively expose latent alignment
issues. Given the low cost of our method, we are able to conduct a large scale
study regarding LLM alignment issues in different worlds. Our results show that
our method outperforms the-state-of-the-art jailbreaking techniques on both
effectiveness and efficiency. In addition, our results indicate that existing
LLMs are extremely vulnerable to nesting worlds and programming language
worlds. They imply that existing alignment training focuses on the real-world
and is lacking in various (virtual) worlds where LLMs can be exploited.

我们发现大型语言模型在不同环境下存在不同程度的对齐问题，并通过构建多个上下文（称为世界）并利用相应的编译器，低成本地暴露潜在的对齐问题，从而进行大规模的对 LLM 对齐问题的研究，结果表明我们的方法在效果和效率上优于现有的破解技术。此外，我们的结果表明，现有的 LLMs 在嵌套世界和编程语言世界中存在极高的漏洞性，这暗示现有的对齐训练偏重于真实世界，对 LLMs 在各种（虚拟）世界中的潜在利用存在缺失。