The recent wave of generative AI has sparked unprecedented global attention,
with both excitement and concern over potentially superhuman levels of
artificial intelligence: models now take only seconds to produce outputs that
would challenge or exceed the capabilities even of expert humans. At the same
time, models still show basic errors in understanding that would not be
expected even in non-expert humans. This presents us with an apparent paradox:
how do we reconcile seemingly superhuman capabilities with the persistence of
errors that few humans would make? In this work, we posit that this tension
reflects a divergence in the configuration of intelligence in today's
generative models relative to intelligence in humans. Specifically, we propose
and test the Generative AI Paradox hypothesis: generative models, having been
trained directly to reproduce expert-like outputs, acquire generative
capabilities that are not contingent upon -- and can therefore exceed -- their
ability to understand those same types of outputs. This contrasts with humans,
for whom basic understanding almost always precedes the ability to generate
expert-level outputs. We test this hypothesis through controlled experiments
analyzing generation vs. understanding in generative models, across both
language and image modalities. Our results show that although models can
outperform humans in generation, they consistently fall short of human
capabilities in measures of understanding, as well as weaker correlation
between generation and understanding performance, and more brittleness to
adversarial inputs. Our findings support the hypothesis that models' generative
capability may not be contingent upon understanding capability, and call for
caution in interpreting artificial intelligence by analogy to human
intelligence.

这项研究揭示了生成型人工智能模型在超人能力生成方面表现出色，但在理解能力方面远远落后于人类，从而提出了 “生成型人工智能悖论” 假说，并通过对语言和图像模式下的控制实验进行了验证。

生成型人工智能的悖论：「它能创造，却未必能理解」

The Generative AI Paradox: "What It Can Create, It May Not Understand"

AI alignment aims to build AI systems that are in accordance with human
intentions and values. With the emergence of AI systems possessing superhuman
capabilities, the potential large-scale risks associated with misaligned
systems become apparent. Hundreds of AI experts and public figures have
expressed their concerns about AI risks, arguing that mitigating the risk of
extinction from AI should be a global priority, alongside other societal-scale
risks such as pandemics and nuclear war. Motivated by the lack of an up-to-date
systematic survey on AI alignment, in this paper, we delve into the core
concepts, methodology, and practice of alignment research. To begin with, we
identify four principles as the key objectives of AI alignment: Robustness,
Interpretability, Controllability, and Ethicality (RICE). We outline the
landscape of current alignment research and decompose them into two key
components: forward alignment and backward alignment. The former aims to make
AI systems aligned via alignment training, while the latter aims to gain
evidence about the systems' alignment and govern them appropriately to avoid
exacerbating misalignment risks. On forward alignment, we discuss how to
conduct learning from various types of feedback (a.k.a., outer alignment) and
how to overcome the distribution shift to avoid goal misgeneralization (a.k.a.,
inner alignment). On backward alignment, we discuss verification techniques
that can tell the degree of value alignment for various AI systems deployed,
which can further improve the assurance of forward alignment outcomes.
Based on this, we also release a constantly updated website featuring
tutorials, collections of papers, blogs, and other learning resources at
this https URL

AI alignment aims to build AI systems in accordance with human intentions and values, addressing the risks of misaligned systems with superhuman capabilities through forward and backward alignment methodologies.