We consider the problem of building a state representation model for control,
in a continual learning setting. As the environment changes, the aim is to
efficiently compress the sensory state's information without losing past
knowledge, and then use Reinforcement Learning on the resulting features for
efficient policy learning. To this end, we propose S-TRIGGER, a general method
for Continual State Representation Learning applicable to Variational
Auto-Encoders and its many variants. The method is based on Generative Replay,
i.e. the use of generated samples to maintain past knowledge. It comes along
with a statistically sound method for environment change detection, which
self-triggers the Generative Replay. Our experiments on VAEs show that
S-TRIGGER learns state representations that allows fast and high-performing
Reinforcement Learning, while avoiding catastrophic forgetting. The resulting
system is capable of autonomously learning new information without using past
data and with a bounded system size. Code for our experiments is attached in
Appendix.

在不断学习的情境下，我们提出了一种名为 S-TRIGGER 的新的方法，通过使用生成的样本来维持过去的知识并使用生成的样本的状态表示，在避免灾难性遗忘的情况下，迅速高效地进行强化学习，同时避免了使用过去数据和有限的系统大小问题。

S-TRIGGER: 自触发生成重放的连续状态表示学习

S-TRIGGER: Continual State Representation Learning via Self-Triggered  Generative Replay

We consider the problem of building a state representation model in a
continual fashion. As the environment changes, the aim is to efficiently
compress the sensory state's information without losing past knowledge. The
learned features are then fed to a Reinforcement Learning algorithm to learn a
policy. We propose to use Variational Auto-Encoders for state representation,
and Generative Replay, i.e. the use of generated samples, to maintain past
knowledge. We also provide a general and statistically sound method for
automatic environment change detection. Our method provides efficient state
representation as well as forward transfer, and avoids catastrophic forgetting.
The resulting model is capable of incrementally learning information without
using past data and with a bounded system size.

本研究针对建立连续学习中的状态表示模型问题展开讨论，为保留过去知识的同时高效压缩感知状态信息，我们提出了使用变分自编码器（Variational Auto-Encoders）作为状态表示，并基于生成样本的生成回放（Generative Replay）技术来保持过去知识。我们还提供了一种统计意义上合理的自动环境变化检测方法。该方法不仅可提供高效的状态表示和前向传递，还可避免灾难性遗忘。最后得出的模型能够在有限系统空间内逐增学习，无需使用过去数据。