Deep reinforcement learning (DRL) has shown significant promise for
uncovering sophisticated control policies that interact in environments with
complicated dynamics, such as stabilizing the magnetohydrodynamics of a tokamak
fusion reactor or minimizing the drag force exerted on an object in a fluid
flow. However, these algorithms require an abundance of training examples and
may become prohibitively expensive for many applications. In addition, the
reliance on deep neural networks often results in an uninterpretable, black-box
policy that may be too computationally expensive to use with certain embedded
systems. Recent advances in sparse dictionary learning, such as the sparse
identification of nonlinear dynamics (SINDy), have shown promise for creating
efficient and interpretable data-driven models in the low-data regime. In this
work we introduce SINDy-RL, a unifying framework for combining SINDy and DRL to
create efficient, interpretable, and trustworthy representations of the
dynamics model, reward function, and control policy. We demonstrate the
effectiveness of our approaches on benchmark control environments and
challenging fluids problems. SINDy-RL achieves comparable performance to
state-of-the-art DRL algorithms using significantly fewer interactions in the
environment and results in an interpretable control policy orders of magnitude
smaller than a deep neural network policy.

介绍了一种将稀疏字典学习和深度强化学习结合起来的 SINDy-RL 框架，用于创建高效、可解释和可信赖的动力学模型、奖励函数和控制策略，并通过在基准控制环境和复杂流体问题上的实验表明其有效性。