Locally supervised learning aims to train a neural network based on a local
estimation of the global loss function at each decoupled module of the network.
Auxiliary networks are typically appended to the modules to approximate the
gradient updates based on the greedy local losses. Despite being advantageous
in terms of parallelism and reduced memory consumption, this paradigm of
training severely degrades the generalization performance of neural networks.
In this paper, we propose Periodically Guided local Learning (PGL), which
reinstates the global objective repetitively into the local-loss based training
of neural networks primarily to enhance the model's generalization capability.
We show that a simple periodic guidance scheme begets significant performance
gains while having a low memory footprint. We conduct extensive experiments on
various datasets and networks to demonstrate the effectiveness of PGL,
especially in the configuration with numerous decoupled modules.

提出一种周期性引导的本地化学习方法，旨在提升神经网络的泛化能力。该方法将全局目标函数反复引入神经网络的本地化损失训练中，并在多个模块的配置中具有重要的性能提升。