The problem of statistical inference in its various forms has been the
subject of decades-long extensive research. Most of the effort has been focused
on characterizing the behavior as a function of the number of available
samples, with far less attention given to the effect of memory limitations on
performance. Recently, this latter topic has drawn much interest in the
engineering and computer science literature. In this survey paper, we attempt
to review the state-of-the-art of statistical inference under memory
constraints in several canonical problems, including hypothesis testing,
parameter estimation, and distribution property testing/estimation. We discuss
the main results in this developing field, and by identifying recurrent themes,
we extract some fundamental building blocks for algorithmic construction, as
well as useful techniques for lower bound derivations.

本文综述了在统计推断中存在的内存限制对性能的影响，包括假设检验、参数估计和分布特性检验 / 估计等几个典型问题，总结了该领域内的主要研究成果，并提取了一些算法构建的基本模块和推导下限的有用技术。

有限记忆下的统计推断：一项调查

Statistical Inference with Limited Memory: A Survey

On-device training has become an increasingly popular approach to machine
learning, enabling models to be trained directly on mobile and edge devices.
However, a major challenge in this area is the limited memory available on
these devices, which can severely restrict the size and complexity of the
models that can be trained. In this systematic survey, we aim to explore the
current state-of-the-art techniques for breaking on-device training memory
walls, focusing on methods that can enable larger and more complex models to be
trained on resource-constrained devices.
Specifically, we first analyze the key factors that contribute to the
phenomenon of memory walls encountered during on-device training. Then, we
present a comprehensive literature review of on-device training, which
addresses the issue of memory limitations. Finally, we summarize on-device
training and highlight the open problems for future research.
By providing a comprehensive overview of these techniques and their
effectiveness in breaking memory walls, we hope to help researchers and
practitioners in this field navigate the rapidly evolving landscape of
on-device training.

本文系统调查了在设备上训练的方法，重点关注可以在资源受限设备上训练更大更复杂的模型的技术，并通过文献综述和总结来解决内存限制和未来研究的问题。

突破设备端训练内存瓶颈：系统调查

Breaking On-device Training Memory Wall: A Systematic Survey

The two main impediments to continual learning are catastrophic forgetting
and memory limitations on the storage of data. To cope with these challenges,
we propose a novel, cognitively-inspired approach which trains autoencoders
with Neural Style Transfer to encode and store images. During training on a new
task, reconstructed images from encoded episodes are replayed in order to avoid
catastrophic forgetting. The loss function for the reconstructed images is
weighted to reduce its effect during classifier training to cope with image
degradation. When the system runs out of memory the encoded episodes are
converted into centroids and covariance matrices, which are used to generate
pseudo-images during classifier training, keeping classifier performance stable
while using less memory. Our approach increases classification accuracy by
13-17% over state-of-the-art methods on benchmark datasets, while requiring 78%
less storage space.

使用自动编码器和神经风格转移的方法来解决持续学习中的灾难性遗忘和存储数据的内存限制问题，并取得了比最先进方法更高的分类准确率和更少的存储空间。

EEC: 学习图像编码和再生，用于持续学习

EEC: Learning to Encode and Regenerate Images for Continual Learning

The two main challenges faced by continual learning approaches are
catastrophic forgetting and memory limitations on the storage of data. To cope
with these challenges, we propose a novel, cognitively-inspired approach which
trains autoencoders with Neural Style Transfer to encode and store images.
Reconstructed images from encoded episodes are replayed when training the
classifier model on a new task to avoid catastrophic forgetting. The loss
function for the reconstructed images is weighted to reduce its effect during
classifier training to cope with image degradation. When the system runs out of
memory the encoded episodes are converted into centroids and covariance
matrices, which are used to generate pseudo-images during classifier training,
keeping classifier performance stable with less memory. Our approach increases
classification accuracy by 13-17% over state-of-the-art methods on benchmark
datasets, while requiring 78% less storage space.

提出了一种新型认知启发式方法，该方法使用神经风格转移训练自编码器进行图像编码和存储，通过重放编码图像来解决连续学习中的灾难性遗忘问题，在分类器训练期间生成伪图像来解决内存限制问题，从而有效提高分类准确率并降低存储空间。