Neural networks are composed of multiple layers arranged in a hierarchical
structure jointly trained with a gradient-based optimization, where the errors
are back-propagated from the last layer back to the first one. At each
optimization step, neurons at a given layer receive feedback from neurons
belonging to higher layers of the hierarchy. In this paper, we propose to
complement this traditional 'between-layer' feedback with additional
'within-layer' feedback to encourage the diversity of the activations within
the same layer. To this end, we measure the pairwise similarity between the
outputs of the neurons and use it to model the layer's overall diversity. We
present an extensive empirical study confirming that the proposed approach
enhances the performance of several state-of-the-art neural network models in
multiple tasks. The code is publically available at
https://github.com/firasl/AAAI-23-WLD-Reg

本文提出一种基于神经网络的层内反馈机制，通过计算神经元输出之间的相似性来增强每一层内的激活多样性，实验证明这种方法能够提高多种任务下多个最新神经网络模型的性能。