On-device learning remains a formidable challenge, especially when dealing
with resource-constrained devices that have limited computational capabilities.
This challenge is primarily rooted in two key issues: first, the memory
available on embedded devices is typically insufficient to accommodate the
memory-intensive back-propagation algorithm, which often relies on
floating-point precision. Second, the development of learning algorithms on
models with extreme quantization levels, such as Binary Neural Networks (BNNs),
is critical due to the drastic reduction in bit representation. In this study,
we propose a solution that combines recent advancements in the field of
Continual Learning (CL) and Binary Neural Networks to enable on-device training
while maintaining competitive performance. Specifically, our approach leverages
binary latent replay (LR) activations and a novel quantization scheme that
significantly reduces the number of bits required for gradient computation. The
experimental validation demonstrates a significant accuracy improvement in
combination with a noticeable reduction in memory requirement, confirming the
suitability of our approach in expanding the practical applications of deep
learning in real-world scenarios.

提出了一种将连续学习和二进制神经网络结合起来的解决方案，同时在设备上进行训练并保持竞争性的性能，该方法利用二进制潜在重播激活和一种新的量化方案，显著减少了梯度计算所需的位数，实验证实了模型的准确性和对内存需求的明显减少，从而扩展了深度学习在实际场景中的应用。