The original vision of AI was re-articulated in 2002 via the term 'Artificial
General Intelligence' or AGI. This vision is to build 'Thinking Machines' -
computer systems that can learn, reason, and solve problems similar to the way
humans do. This is in stark contrast to the 'Narrow AI' approach practiced by
almost everyone in the field over the many decades. While several large-scale
efforts have nominally been working on AGI (most notably DeepMind), the field
of pure focused AGI development has not been well funded or promoted. This is
surprising given the fantastic value that true AGI can bestow on humanity. In
addition to the dearth of effort in this field, there are also several
theoretical and methodical missteps that are hampering progress. We highlight
why purely statistical approaches are unlikely to lead to AGI, and identify
several crucial cognitive abilities required to achieve human-like adaptability
and autonomous learning. We conclude with a survey of socio-technical factors
that have undoubtedly slowed progress towards AGI.

人工通用智能（AGI）是构建 “思考机器” 的愿景，与 “狭义人工智能” 方法形成鲜明对比。本研究指出了纯统计方法无法实现 AGI，同时确定了实现类似人类适应能力和自主学习所需的关键认知能力，并总结了阻碍 AGI 进展的社会技术因素。

为何我们尚未拥有通用人工智能

Why We Don't Have AGI Yet

How to evaluate Artificial General Intelligence (AGI) is a critical problem
that is discussed and unsolved for a long period. In the research of narrow AI,
this seems not a severe problem, since researchers in that field focus on some
specific problems as well as one or some aspects of cognition, and the criteria
for evaluation are explicitly defined. By contrast, an AGI agent should solve
problems that are never-encountered by both agents and developers. However,
once a developer tests and debugs the agent with a problem, the
never-encountered problem becomes the encountered problem, as a result, the
problem is solved by the developers to some extent, exploiting their
experience, rather than the agents. This conflict, as we call the trap of
developers' experience, leads to that this kind of problems is probably hard to
become an acknowledged criterion. In this paper, we propose an evaluation
method named Artificial Open World, aiming to jump out of the trap. The
intuition is that most of the experience in the actual world should not be
necessary to be applied to the artificial world, and the world should be open
in some sense, such that developers are unable to perceive the world and solve
problems by themselves before testing, though after that they are allowed to
check all the data. The world is generated in a similar way as the actual
world, and a general form of problems is proposed. A metric is proposed aiming
to quantify the progress of research. This paper describes the conceptual
design of the Artificial Open World, though the formalization and the
implementation are left to the future.

本文提出了一种名为 “人工开放世界” 的评估方法，旨在解决开发者经验陷阱问题，该方法通过创建一个人工世界，以一种类似实际世界的方式生成问题并提出度量标准来量化研究进展。

评估 AGI 的人工开放世界：一个概念设计

Artificial Open World for Evaluating AGI: a Conceptual Design

In this work, we present and analyze reported failures of artificially
intelligent systems and extrapolate our analysis to future AIs. We suggest that
both the frequency and the seriousness of future AI failures will steadily
increase. AI Safety can be improved based on ideas developed by cybersecurity
experts. For narrow AIs safety failures are at the same, moderate, level of
criticality as in cybersecurity, however for general AI, failures have a
fundamentally different impact. A single failure of a superintelligent system
may cause a catastrophic event without a chance for recovery. The goal of
cybersecurity is to reduce the number of successful attacks on the system; the
goal of AI Safety is to make sure zero attacks succeed in bypassing the safety
mechanisms. Unfortunately, such a level of performance is unachievable. Every
security system will eventually fail; there is no such thing as a 100% secure
system.

研究人工智能的安全性，认为未来人工智能的故障频率和严重性将不断增加，并将其与网络安全相比较。对于一般人工智能系统失误的影响将比狭窄人工智能系统更深远，建议应加强其安全性以确保零失误的发生，尽管这是不可能的。