We introduce a variational approach to learning and inference of temporally hierarchical structure and representation for sequential data. We propose the Variational Temporal Abstraction (VTA), a hierarchical recurrent state space model that can infer the latent temporal structure and thus perform the stochastic state transition hierarchically. We also propose to apply this model to implement the jumpy-imagination ability in imagination-augmented agent-learning in order to improve the efficiency of the imagination. In experiments, we demonstrate that our proposed method can model 2D and 3D visual sequence datasets with interpretable temporal structure discovery and that its application to jumpy imagination enables more efficient agent-learning in a 3D navigation task.

本文介绍了一种用于序列数据学习和推理的变分方法，即基于变分时间抽象（VTA）的分层循环状态空间模型，该模型能够推断潜在的时间结构并因此进行随机状态转换分层；同时本文还提出了将该模型应用于增强想象学习中的跳跃想象能力实现，并且实验验证表明本文提出的方法能够对2D和3D视觉序列数据集进行可解释性时间结构发现并且应用于多样化的想象能力可以有效提高3D导航任务中的智能体学习效率。