Connectionist models such as neural networks suffer from catastrophic
forgetting. In this work, we study this problem from the perspective of
information theory and define forgetting as the increase of description lengths
of previous data when they are compressed with a sequentially learned model. In
addition, we show that continual learning approaches based on variational
posterior approximation and generative replay can be considered as
approximations to two prequential coding methods in compression, namely, the
Bayesian mixture code and maximum likelihood (ML) plug-in code. We compare
these approaches in terms of both compression and forgetting and empirically
study the reasons that limit the performance of continual learning methods
based on variational posterior approximation. To address these limitations, we
propose a new continual learning method that combines ML plug-in and Bayesian
mixture codes.

通过信息理论的角度研究连接主义模型（如神经网络）的灾难性遗忘问题，提出连续学习方法是在压缩中 Bayesian 混合编码和最大似然（ML）插入编码的近似，并比较其性能和限制，最终提出了一种结合了 ML 插入和 Bayesian 混合编码的新的连续学习方法。

从压缩的角度看持续学习

Continual Learning from the Perspective of Compression

In order to meet the diverse challenges in solving many real-world problems,
an intelligent agent has to be able to dynamically construct a model of its
environment. Objects facilitate the modular reuse of prior knowledge and the
combinatorial construction of such models. In this work, we argue that
dynamically bound features (objects) do not simply emerge in connectionist
models of the world. We identify several requirements that need to be fulfilled
in overcoming this limitation and highlight corresponding inductive biases.

本文论述在智能代理的建模过程中，对象化有助于先前知识的模块化再使用和组合构建模型。但针对此限制，文章探讨了在连接主义模型中动态绑定特征（对象）的产生需要满足的需求以及相应的归纳偏差。

基于模型的强化学习中对象和系统化概括的视角

A Perspective on Objects and Systematic Generalization in Model-Based RL

The study and understanding of human behaviour is relevant to computer
science, artificial intelligence, neural computation, cognitive science,
philosophy, psychology, and several other areas. Presupposing cognition as
basis of behaviour, among the most prominent tools in the modelling of
behaviour are computational-logic systems, connectionist models of cognition,
and models of uncertainty. Recent studies in cognitive science, artificial
intelligence, and psychology have produced a number of cognitive models of
reasoning, learning, and language that are underpinned by computation. In
addition, efforts in computer science research have led to the development of
cognitive computational systems integrating machine learning and automated
reasoning. Such systems have shown promise in a range of applications,
including computational biology, fault diagnosis, training and assessment in
simulators, and software verification. This joint survey reviews the personal
ideas and views of several researchers on neural-symbolic learning and
reasoning. The article is organised in three parts: Firstly, we frame the scope
and goals of neural-symbolic computation and have a look at the theoretical
foundations. We then proceed to describe the realisations of neural-symbolic
computation, systems, and applications. Finally we present the challenges
facing the area and avenues for further research.

这篇文章介绍了关于认知科学、人工智能、神经计算的研究，特别是神经符号学习和推理的最新进展，从理论到应用和挑战，以及展望未来。