Backpropagation (BP) has been a successful optimization technique for deep
learning models. However, its limitations, such as backward- and
update-locking, and its biological implausibility, hinder the concurrent
updating of layers and do not mimic the local learning processes observed in
the human brain. To address these issues, recent research has suggested using
local error signals to asynchronously train network blocks. However, this
approach often involves extensive trial-and-error iterations to determine the
best configuration for local training. This includes decisions on how to
decouple network blocks and which auxiliary networks to use for each block. In
our work, we introduce a novel BP-free approach: a block-wise BP-free (BWBPF)
neural network that leverages local error signals to optimize distinct
sub-neural networks separately, where the global loss is only responsible for
updating the output layer. The local error signals used in the BP-free model
can be computed in parallel, enabling a potential speed-up in the weight update
process through parallel implementation. Our experimental results consistently
show that this approach can identify transferable decoupled architectures for
VGG and ResNet variations, outperforming models trained with end-to-end
backpropagation and other state-of-the-art block-wise learning techniques on
datasets such as CIFAR-10 and Tiny-ImageNet. The code is released at
this https URL

通过引入一种新的无反向传播的方法，本研究提出了一个基于局部误差信号的块状神经网络，可以分别优化不同的子神经网络，并通过并行计算局部误差信号来加速权重更新过程，取得了比使用端到端反向传播和其他最先进的块状学习技术更好的性能，特别是在 CIFAR-10 和 Tiny-ImageNet 数据集上。