During the first part of life, the brain develops while it learns through a
process called synaptogenesis. The neurons, growing and interacting with each
other, create synapses. However, eventually the brain prunes those synapses.
While previous work focused on learning and pruning independently, in this work
we propose a biologically plausible model that, thanks to a combination of
Hebbian learning and pruning, aims to simulate the synaptogenesis process. In
this way, while learning how to solve the task, the agent translates its
experience into a particular network structure. Namely, the network structure
builds itself during the execution of the task. We call this approach
Self-building Neural Network (SBNN). We compare our proposed SBNN with
traditional neural networks (NNs) over three classical control tasks from
OpenAI. The results show that our model performs generally better than
traditional NNs. Moreover, we observe that the performance decay while
increasing the pruning rate is smaller in our model than with NNs. Finally, we
perform a validation test, testing the models over tasks unseen during the
learning phase. In this case, the results show that SBNNs can adapt to new
tasks better than the traditional NNs, especially when over $80\%$ of the
weights are pruned.

为了在任务执行期间自动构建神经网络结构提出了一种自构建神经网络（SBNN）。 该模型采用 Hebbian 学习和修剪相结合的方法，以拟合神经元的突触形成过程。 在与传统神经网络的比较中，结果表明其表现通常优于传统神经网络，并且在验证测试中 SBNNs 比传统神经网络更适应新任务，特别是在注重权重修剪的情况下。

自构建神经网络

Self-building Neural Networks

Spiking Neural Networks (SNNs) have been attached great importance due to
their biological plausibility and high energy-efficiency on neuromorphic chips.
As these chips are usually resource-constrained, the compression of SNNs is
thus crucial along the road of practical use of SNNs. Most existing methods
directly apply pruning approaches in artificial neural networks (ANNs) to SNNs,
which ignore the difference between ANNs and SNNs, thus limiting the
performance of the pruned SNNs. Besides, these methods are only suitable for
shallow SNNs. In this paper, inspired by synaptogenesis and synapse elimination
in the neural system, we propose gradient rewiring (Grad R), a joint learning
algorithm of connectivity and weight for SNNs, that enables us to seamlessly
optimize network structure without retraining. Our key innovation is to
redefine the gradient to a new synaptic parameter, allowing better exploration
of network structures by taking full advantage of the competition between
pruning and regrowth of connections. The experimental results show that the
proposed method achieves minimal loss of SNNs' performance on MNIST and
CIFAR-10 dataset so far. Moreover, it reaches a $\sim$3.5% accuracy loss under
unprecedented 0.73% connectivity, which reveals remarkable structure refining
capability in SNNs. Our work suggests that there exists extremely high
redundancy in deep SNNs. Our codes are available at
this https URL

本文提出了一种基于神经系统突触发生和消失机制的梯度重连算法，用于 SNNs 中的连接性和权重优化，实现了对网络结构的优化而无需重新训练，实验结果表明，该方法能够在 MNIST 和 CIFAR-10 数据集上实现对 SNNs 性能的最小损失，并且在 0.73% 的连接性下，只有约 3.5% 的精度损失，具有优秀的结构优化能力。