A challenging problem in task-free continual learning is the online selection
of a representative replay memory from data streams. In this work, we
investigate the online memory selection problem from an information-theoretic
perspective. To gather the most information, we propose the \textit{surprise}
and the \textit{learnability} criteria to pick informative points and to avoid
outliers. We present a Bayesian model to compute the criteria efficiently by
exploiting rank-one matrix structures. We demonstrate that these criteria
encourage selecting informative points in a greedy algorithm for online memory
selection. Furthermore, by identifying the importance of \textit{the timing to
update the memory}, we introduce a stochastic information-theoretic reservoir
sampler (InfoRS), which conducts sampling among selective points with high
information. Compared to reservoir sampling, InfoRS demonstrates improved
robustness against data imbalance. Finally, empirical performances over
continual learning benchmarks manifest its efficiency and efficacy.

本文是关于使用信息论从数据流中选择回放内存的在线选择问题的研究，提出了 Surprise 和 Learnability 准则，并使用贝叶斯模型高效地计算这些准则，进一步引入信息熵随机采样器（InfoRS）来选择高信息熵的点进行采样。在连续学习基准测试中，验证了它的效率和功效。