Widespread deployment of societal-scale machine learning systems necessitates
a thorough understanding of the resulting long-term effects these systems have
on their environment, including loss of trustworthiness, bias amplification,
and violation of AI safety requirements. We introduce a repeated learning
process to jointly describe several phenomena attributed to unintended hidden
feedback loops, such as error amplification, induced concept drift, echo
chambers and others. The process comprises the entire cycle of obtaining the
data, training the predictive model, and delivering predictions to end-users
within a single mathematical model. A distinctive feature of such repeated
learning setting is that the state of the environment becomes causally
dependent on the learner itself over time, thus violating the usual assumptions
about the data distribution. We present a novel dynamical systems model of the
repeated learning process and prove the limiting set of probability
distributions for positive and negative feedback loop modes of the system
operation. We conduct a series of computational experiments using an exemplary
supervised learning problem on two synthetic data sets. The results of the
experiments correspond to the theoretical predictions derived from the
dynamical model. Our results demonstrate the feasibility of the proposed
approach for studying the repeated learning processes in machine learning
systems and open a range of opportunities for further research in the area.

广泛部署面向社会规模的机器学习系统需要深入了解这些系统对环境产生的长期影响，包括失去可信度、偏见放大和违反 AI 安全要求。我们引入了一种重复学习过程，共同描述了一些与意外隐藏反馈循环相关的现象，如错误放大、引发的概念漂移、回声室等。该过程包括在单个数学模型中获取数据、训练预测模型和向最终用户传递预测的整个周期。这种重复学习设置的一个显著特点是，随着时间的推移，环境的状态变得因果关联于学习者本身，从而违背了有关数据分布的通常假设。我们提出了重复学习过程的一种新颖动力系统模型，并证明了系统运行的正反馈和负反馈环模式的概率分布极限集。我们使用两个合成数据集上的示例监督学习问题进行了一系列计算实验。实验结果与从动力模型推导出的理论预测相一致。我们的结果证明了在机器学习系统中研究重复学习过程所提出方法的可行性，并为进一步研究提供了一系列机会。