The ability to learn good representations of states is essential for solving
large reinforcement learning problems, where exploration, generalization, and
transfer are particularly challenging. The Laplacian representation is a
promising approach to address these problems by inducing intrinsic rewards for
temporally-extended action discovery and reward shaping, and informative state
encoding. To obtain the Laplacian representation one needs to compute the
eigensystem of the graph Laplacian, which is often approximated through
optimization objectives compatible with deep learning approaches. These
approximations, however, depend on hyperparameters that are impossible to tune
efficiently, converge to arbitrary rotations of the desired eigenvectors, and
are unable to accurately recover the corresponding eigenvalues. In this paper
we introduce a theoretically sound objective and corresponding optimization
algorithm for approximating the Laplacian representation. Our approach
naturally recovers both the true eigenvectors and eigenvalues while eliminating
the hyperparameter dependence of previous approximations. We provide
theoretical guarantees for our method and we show that those results translate
empirically into robust learning across multiple environments.

我们介绍了一种用于近似拉普拉斯表示的理论上可靠的目标及相应的优化算法，通过消除先前逼近的超参数依赖性，我们能够自然地恢复真实特征向量和特征值。我们提供了我们方法的理论保证，并展示这些结果在多个环境中的鲁棒学习表现。