Embed-to-control (E2C) is a model for solving high-dimensional optimal control problems by combining variational auto-encoders with locally-optimal controllers. However, the current E2C model suffers from two major drawbacks: 1) its objective function does not correspond to the likelihood of the data sequence and 2) the variational encoder used for embedding typically has large variational approximation error, especially when there is noise in the system dynamics. In this paper, we present a new model for learning robust locally-linear controllable embedding (RCE). Our model directly estimates the predictive conditional density of the future observation given the current one, while introducing the bottleneck between the current and future observations. Although the bottleneck provides a natural embedding candidate for control, our RCE model introduces additional specific structures in the generative graphical model so that the model dynamics can be robustly linearized. We also propose a principled variational approximation of the embedding posterior that takes the future observation into account, and thus, makes the variational approximation more robust against the noise. Experimental results demonstrate that RCE outperforms the existing E2C model, and does so significantly in the regime where the underlying dynamics is noisy.

本文提出了一种新的方法——学习具有鲁棒性的局部线性可控嵌入（RCE）模型，该模型可以直接估计给定当前观测下未来观测的预测条件密度，并引入瓶颈结构使得模型动力学具有鲁棒的可线性化特性，同时提出了一种有原则的变分近似方法，使得变分近似更加鲁棒。实验结果表明，与Embed-to-Control（E2C）模型相比，RCE模型具有更好的鲁棒性和预测性能，尤其在系统动力学存在噪声的情况下表现显著。

鲁棒局部线性可控嵌入