A core challenge in the development of increasingly capable AI systems is to
make them safe and reliable by ensuring their behaviour is consistent with
human values. This challenge, known as the alignment problem, does not merely
apply to hypothetical future AI systems that may pose catastrophic risks; it
already applies to current systems, such as large language models, whose
potential for harm is rapidly increasing. In this paper, I assess whether we
are on track to solve the alignment problem for large language models, and what
that means for the safety of future AI systems. I argue that existing
strategies for alignment are insufficient, because large language models remain
vulnerable to adversarial attacks that can reliably elicit unsafe behaviour. I
offer an explanation of this lingering vulnerability on which it is not simply
a contingent limitation of current language models, but has deep technical ties
to a crucial aspect of what makes these models useful and versatile in the
first place -- namely, their remarkable aptitude to learn "in context" directly
from user instructions. It follows that the alignment problem is not only
unsolved for current AI systems, but may be intrinsically difficult to solve
without severely undermining their capabilities. Furthermore, this assessment
raises concerns about the prospect of ensuring the safety of future and more
capable AI systems.

当前的大型语言模型仍然容易受到对抗攻击，使其表现出不安全的行为，这一基本问题不仅对当前的人工智能系统尚未解决，而且在不严重削弱其能力的情况下可能难以解决，同时也对未来和更具能力的人工智能系统的安全性提出了担忧。