In artificial neural networks, the activation dynamics of non-trainable
variables is strongly coupled to the learning dynamics of trainable variables.
During the activation pass, the boundary neurons (e.g., input neurons) are
mapped to the bulk neurons (e.g., hidden neurons), and during the learning
pass, both bulk and boundary neurons are mapped to changes in trainable
variables (e.g., weights and biases). For example, in feed-forward neural
networks, forward propagation is the activation pass and backward propagation
is the learning pass. We show that a composition of the two maps establishes a
duality map between a subspace of non-trainable boundary variables (e.g.,
dataset) and a tangent subspace of trainable variables (i.e., learning). In
general, the dataset-learning duality is a complex non-linear map between
high-dimensional spaces, but in a learning equilibrium, the problem can be
linearized and reduced to many weakly coupled one-dimensional problems. We use
the duality to study the emergence of criticality, or the power-law
distributions of fluctuations of the trainable variables. In particular, we
show that criticality can emerge in the learning system even from the dataset
in a non-critical state, and that the power-law distribution can be modified by
changing either the activation function or the loss function.

在人工神经网络中，非可训练变量的激活动力学与可训练变量的学习动力学强烈耦合。研究表明通过两种映射的组合建立了非可训练边界变量的子空间（例如数据集）与可训练变量的切向子空间（即学习）之间的对偶映射，可以研究临界性的出现以及可训练变量波动的幂律分布的特性。