Sample-efficiency and reliability remain major bottlenecks toward wide
adoption of reinforcement learning algorithms in continuous settings with
high-dimensional perceptual inputs. Toward addressing these challenges, we
introduce a new theoretical framework, RichCLD (Rich-Observation RL with
Continuous Latent Dynamics), in which the agent performs control based on
high-dimensional observations, but the environment is governed by
low-dimensional latent states and Lipschitz continuous dynamics. Our main
contribution is a new algorithm for this setting that is provably statistically
and computationally efficient. The core of our algorithm is a new
representation learning objective; we show that prior representation learning
schemes tailored to discrete dynamics do not naturally extend to the continuous
setting. Our new objective is amenable to practical implementation, and
empirically, we find that it compares favorably to prior schemes in a standard
evaluation protocol. We further provide several insights into the statistical
complexity of the RichCLD framework, in particular proving that certain notions
of Lipschitzness that admit sample-efficient learning in the absence of rich
observations are insufficient in the rich-observation setting.

針對連續環境中高維感知輸入的強化學習算法的樣本效率和可靠性問題，本研究提出了一種新的理論框架 RichCLD（基於豐富觀測的連續潛在動力學強化學習），其中代理基於高維觀測進行控制，但環境卻由低維潛在狀態和 Lipschitz 連續動力學所控制。我們的主要貢獻是提出了一種新的算法，可以在統計和計算效率方面獲得證明。我們的算法的核心是一個新的表示學習目標；我們表明，適用於離散動力學的先前表示學習方案不自然地擴展到連續環境。我們的新目標易於實施，實驗結果顯示它在標准評估協議中相比先前方案具有競爭優勢。此外，我們對 RichCLD 框架的統計複雜性提供了一些見解，特別是證明了在豐富觀測缺失情況下的樣本效率學習所需的某些 Lipschitz 性質在豐富觀測情況下是不夠的。