Continual learning is the sequential learning of different tasks by a machine
learning model. Continual learning is known to be hindered by catastrophic
interference or forgetting, i.e. rapid unlearning of earlier learned tasks when
new tasks are learned. Despite their practical success, artificial neural
networks (ANNs) are prone to catastrophic interference. This study analyses how
gradient descent and overlapping representations between distant input points
lead to distal interference and catastrophic interference. Distal interference
refers to the phenomenon where training a model on a subset of the domain leads
to non-local changes on other subsets of the domain. This study shows that
uniformly trainable models without distal interference must be exponentially
large. A novel antisymmetric bounded exponential layer B-spline ANN
architecture named ABEL-Spline is proposed that can approximate any continuous
function, is uniformly trainable, has polynomial computational complexity, and
provides some guarantees for distal interference. Experiments are presented to
demonstrate the theoretical properties of ABEL-Splines. ABEL-Splines are also
evaluated on benchmark regression problems. It is concluded that the weaker
distal interference guarantees in ABEL-Splines are insufficient for model-only
continual learning. It is conjectured that continual learning with polynomial
complexity models requires augmentation of the training data or algorithm.

连续学习中的灾难性干扰和远程干扰分析，提出一种能近似任何连续函数、具有多项式计算复杂度、并且对远程干扰提供保证的新型人工神经网络结构 ABAL-Spline。通过实验证明 ABAL-Spline 在理论上具有的性质，并在基准回归问题上进行评估，得出使用多项式复杂度模型进行连续学习需要对训练数据或算法进行增强的推测。