Developmental plasticity plays a prominent role in shaping the brain's
structure during ongoing learning in response to dynamically changing
environments. However, the existing network compression methods for deep
artificial neural networks (ANNs) and spiking neural networks (SNNs) draw
little inspiration from the brain's developmental plasticity mechanisms, thus
limiting their ability to learn efficiently, rapidly, and accurately. This
paper proposed a developmental plasticity-inspired adaptive pruning (DPAP)
method, with inspiration from the adaptive developmental pruning of dendritic
spines, synapses, and neurons according to the "use it or lose it, gradually
decay" principle. The proposed DPAP model considers multiple biologically
realistic mechanisms (such as dendritic spine dynamic plasticity,
activity-dependent neural spiking trace, and local synaptic plasticity), with
the addition of an adaptive pruning strategy, so that the network structure can
be dynamically optimized during learning without any pre-training and
retraining. We demonstrated that the proposed DPAP method applied to deep ANNs
and SNNs could learn efficient network architectures. Extensive comparative
experiments show consistent and remarkable performance and speed boost with the
extremely compressed networks on a diverse set of benchmark tasks, especially
neuromorphic datasets for SNNs. This work explores how developmental plasticity
enables the complex deep networks to gradually evolve into brain-like efficient
and compact structures, eventually achieving state-of-the-art (SOTA)
performance for biologically realistic SNNs.

本文提出了一种发育可塑性启发的自适应剪枝方法，可以动态优化神经网络结构，使其学习效率和速度得到持续和显著的提高，尤其是对于神经形态学数据集的脉冲神经网络而言。