Machines that can replicate human intelligence with type 2 reasoning
capabilities should be able to reason at multiple levels of spatio-temporal
abstractions and scales using internal world models. Devising formalisms to
develop such internal world models, which accurately reflect the causal
hierarchies inherent in the dynamics of the real world, is a critical research
challenge in the domains of artificial intelligence and machine learning. This
thesis identifies several limitations with the prevalent use of state space
models (SSMs) as internal world models and propose two new probabilistic
formalisms namely Hidden-Parameter SSMs and Multi-Time Scale SSMs to address
these drawbacks. The structure of graphical models in both formalisms
facilitates scalable exact probabilistic inference using belief propagation, as
well as end-to-end learning via backpropagation through time. This approach
permits the development of scalable, adaptive hierarchical world models capable
of representing nonstationary dynamics across multiple temporal abstractions
and scales. Moreover, these probabilistic formalisms integrate the concept of
uncertainty in world states, thus improving the system's capacity to emulate
the stochastic nature of the real world and quantify the confidence in its
predictions. The thesis also discuss how these formalisms are in line with
related neuroscience literature on Bayesian brain hypothesis and predicitive
processing. Our experiments on various real and simulated robots demonstrate
that our formalisms can match and in many cases exceed the performance of
contemporary transformer variants in making long-range future predictions. We
conclude the thesis by reflecting on the limitations of our current models and
suggesting directions for future research.

使用类型 2 推理能力的模拟人类智能的机器应该能够使用内部世界模型在多个层次的时空抽象和尺度上进行推理。本论文旨在解决人工智能和机器学习领域中存在的内部世界模型研究挑战，并提出了两种新的概率形式化方法，即隐藏参数状态空间模型和多时间尺度状态空间模型，以解决传统状态空间模型的局限性。这些概率形式化方法能够开发可扩展、自适应的分层世界模型，能够表示多个时间抽象和尺度上的非平稳动力学，并结合了世界状态的不确定性概念，从而提高系统模拟现实世界的随机性和预测结果的置信度。论文还讨论了这些形式化方法与贝叶斯大脑假设和预测处理的相关神经科学文献的一致性。我们对各种真实和模拟机器人的实验表明，我们的形式化方法在长期未来预测方面的性能与当代变压器变种相匹配，并在许多情况下超越其性能。最后，我们对当前模型的局限性进行了总结，并提出了未来研究的方向。

利用层次时间抽象的学习世界模型：概率的视角

Learning World Models With Hierarchical Temporal Abstractions: A  Probabilistic Perspective

Predictive coding has emerged as a prominent model of how the brain learns
through predictions, anticipating the importance accorded to predictive
learning in recent AI architectures such as transformers. Here we propose a new
framework for predictive coding called active predictive coding which can learn
hierarchical world models and solve two radically different open problems in
AI: (1) how do we learn compositional representations, e.g., part-whole
hierarchies, for equivariant vision? and (2) how do we solve large-scale
planning problems, which are hard for traditional reinforcement learning, by
composing complex action sequences from primitive policies? Our approach
exploits hypernetworks, self-supervised learning and reinforcement learning to
learn hierarchical world models that combine task-invariant state transition
networks and task-dependent policy networks at multiple abstraction levels. We
demonstrate the viability of our approach on a variety of vision datasets
(MNIST, FashionMNIST, Omniglot) as well as on a scalable hierarchical planning
problem. Our results represent, to our knowledge, the first demonstration of a
unified solution to the part-whole learning problem posed by Hinton, the nested
reference frames problem posed by Hawkins, and the integrated state-action
hierarchy learning problem in reinforcement learning.

提出了一种新的预测编码框架，称为主动预测编码，利用超网络、自监督学习和强化学习来学习层次化的世界模型，并成功解决了细分视觉和复杂动作序列组合等多个问题。