Backpropagation (BP), a foundational algorithm for training artificial neural
networks, predominates in contemporary deep learning. Although highly
successful, it is often considered biologically implausible. A significant
limitation arises from the need for precise symmetry between connections in the
backward and forward pathways to backpropagate gradient signals accurately,
which is not observed in biological brains. Researchers have proposed several
algorithms to alleviate this symmetry constraint, such as feedback alignment
and direct feedback alignment. However, their divergence from backpropagation
dynamics presents challenges, particularly in deeper networks and convolutional
layers. Here we introduce the Product Feedback Alignment (PFA) algorithm. Our
findings demonstrate that PFA closely approximates BP and achieves comparable
performance in deep convolutional networks while avoiding explicit weight
symmetry. Our results offer a novel solution to the longstanding weight
symmetry problem, leading to more biologically plausible learning in deep
convolutional networks compared to earlier methods.

提出了 Product Feedback Alignment（PFA）算法，它在避免显式的权重对称的同时，几乎与反向传播（BP）算法相同，能够在深度卷积网络中实现可比较的性能，从而提供了对长期存在的权重对称问题的新解决方案，实现了更加符合生物合理性的深度卷积网络学习。

不对称权重的深度学习

Deep Learning without Weight Symmetry

The backpropagation algorithm has experienced remarkable success in training
large-scale artificial neural networks, however, its biological-plausibility is
disputed, and it remains an open question whether the brain employs supervised
learning mechanisms akin to it. Here, we propose correlative information
maximization between layer activations as an alternative normative approach to
describe the signal propagation in biological neural networks in both forward
and backward directions. This new framework addresses many concerns about the
biological-plausibility of conventional artificial neural networks and the
backpropagation algorithm. The coordinate descent-based optimization of the
corresponding objective, combined with the mean square error loss function for
fitting labeled supervision data, gives rise to a neural network structure that
emulates a more biologically realistic network of multi-compartment pyramidal
neurons with dendritic processing and lateral inhibitory neurons. Furthermore,
our approach provides a natural resolution to the weight symmetry problem
between forward and backward signal propagation paths, a significant critique
against the plausibility of the conventional backpropagation algorithm. This is
achieved by leveraging two alternative, yet equivalent forms of the correlative
mutual information objective. These alternatives intrinsically lead to forward
and backward prediction networks without weight symmetry issues, providing a
compelling solution to this long-standing challenge.

提出了一种新的神经网络层间激活的相关信息最大化的正式框架，从而解决了传统反向传播算法的生物学上的可信度问题，并提供了一种自然的权重对称问题的解决方案。