Advances in reinforcement learning have led to its successful application in
complex tasks with continuous state and action spaces. Despite these advances
in practice, most theoretical work pertains to finite state and action spaces.
We propose building a theoretical understanding of continuous state and action
spaces by employing a geometric lens. Central to our work is the idea that the
transition dynamics induce a low dimensional manifold of reachable states
embedded in the high-dimensional nominal state space. We prove that, under
certain conditions, the dimensionality of this manifold is at most the
dimensionality of the action space plus one. This is the first result of its
kind, linking the geometry of the state space to the dimensionality of the
action space. We empirically corroborate this upper bound for four MuJoCo
environments. We further demonstrate the applicability of our result by
learning a policy in this low dimensional representation. To do so we introduce
an algorithm that learns a mapping to a low dimensional representation, as a
narrow hidden layer of a deep neural network, in tandem with the policy using
DDPG. Our experiments show that a policy learnt this way perform on par or
better for four MuJoCo control suite tasks.

使用几何镜头建立对连续状态和动作空间的理论理解，以证明可达状态的低维流形的维度最多为动作空间维度加一，并且通过使用 DDPG 的深度神经网络训练出的低维度表示策略学习表现良好。

连续状态和行动空间中强化学习的几何学

On the Geometry of Reinforcement Learning in Continuous State and Action Spaces

Even after decades of research, dynamic scene background reconstruction and
foreground object segmentation are still considered as open problems due
various challenges such as illumination changes, camera movements, or
background noise caused by air turbulence or moving trees. We propose in this
paper to model the background of a frame sequence as a low dimensional manifold
using an autoencoder and compare the reconstructed background provided by this
autoencoder with the original image to compute the foreground/background
segmentation masks. The main novelty of the proposed model is that the
autoencoder is also trained to predict the background noise, which allows to
compute for each frame a pixel-dependent threshold to perform the foreground
segmentation. Although the proposed model does not use any temporal or motion
information, it exceeds the state of the art for unsupervised background
subtraction on the CDnet 2014 and LASIESTA datasets, with a significant
improvement on videos where the camera is moving. It is also able to perform
background reconstruction on some non-video image datasets.

本文提出了一种使用自编码器模拟场景背景的低维流形，通过与原始图像进行对比实现前景 / 背景分割，自编码器同时用于预测背景噪音，从而计算每一帧的像素相关阈值进行前景分割。该模型在 CDNet 2014 和 LASIESTA 数据集上超越了现有技术，且能够在某些非视频图像数据集上实现背景重构。