Artificial general intelligence (AGI) may herald our extinction, according to
AI safety research. Yet claims regarding AGI must rely upon mathematical
formalisms -- theoretical agents we may analyse or attempt to build. AIXI
appears to be the only such formalism supported by proof that its behaviour is
optimal, a consequence of its use of compression as a proxy for intelligence.
Unfortunately, AIXI is incomputable and claims regarding its behaviour highly
subjective. We argue that this is because AIXI formalises cognition as taking
place in isolation from the environment in which goals are pursued (Cartesian
dualism). We propose an alternative, supported by proof and experiment, which
overcomes these problems. Integrating research from cognitive science with AI,
we formalise an enactive model of learning and reasoning to address the problem
of subjectivity. This allows us to formulate a different proxy for
intelligence, called weakness, which addresses the problem of incomputability.
We prove optimal behaviour is attained when weakness is maximised. This proof
is supplemented by experimental results comparing weakness and description
length (the closest analogue to compression possible without reintroducing
subjectivity). Weakness outperforms description length, suggesting it is a
better proxy. Furthermore we show that, if cognition is enactive, then
minimisation of description length is neither necessary nor sufficient to
attain optimal performance, undermining the notion that compression is closely
related to intelligence. However, there remain open questions regarding the
implementation of scale-able AGI. In the short term, these results may be best
utilised to improve the performance of existing systems. For example, our
results explain why Deepmind's Apperception Engine is able to generalise
effectively, and how to replicate that performance by maximising weakness.

提出一种基于经验的理论框架，称为 “实用弱点原则”，用于处理认知科学和人工智能中的主观性问题，并将其与 AIXI 进行比较，显示出在弱点最大化的情况下会取得最优的行为结果。