Existing methods for optimal control struggle to deal with the complexity
commonly encountered in real-world systems, including dimensionality, process
error, model bias and data heterogeneity. Instead of tackling these system
complexities directly, researchers have typically sought to simplify models to
fit optimal control methods. But when is the optimal solution to an
approximate, stylized model better than an approximate solution to a more
accurate model? While this question has largely gone unanswered owing to the
difficulty of finding even approximate solutions for complex models, recent
algorithmic and computational advances in deep reinforcement learning (DRL)
might finally allow us to address these questions. DRL methods have to date
been applied primarily in the context of games or robotic mechanics, which
operate under precisely known rules. Here, we demonstrate the ability for DRL
algorithms using deep neural networks to successfully approximate solutions
(the "policy function" or control rule) in a non-linear three-variable model
for a fishery without knowing or ever attempting to infer a model for the
process itself. We find that the reinforcement learning agent discovers an
effective simplification of the problem to obtain an interpretable control
rule. We show that the policy obtained with DRL is both more profitable and
more sustainable than any constant mortality policy -- the standard family of
policies considered in fishery management.

现有的最优控制方法在处理真实系统中常遇到的复杂性（包括维度、过程误差、模型偏差和数据异质性）时遇到困难。最优控制方法通常寻求简化模型以适应，但何时适用于近似的、简化的模型的最优解胜过适用于更准确的模型的近似解尚未得到充分回答。深度强化学习（DRL）中的算法和计算进展为我们解决这些问题提供了可能性。DRL 方法迄今主要应用于游戏或机器人机械学中，在确切已知规则下运行。我们在没有了解或企图推断进程模型的情况下，展示了使用深度神经网络进行 DRL 算法的能力，成功近似求解了一个非线性三变量渔业模型的解（“策略函数” 或控制规则）。我们发现强化学习代理通过简化问题来获得可解释的控制规则。我们展示了 DRL 所获得的策略在渔业管理中既更具利润性，也更具可持续性，胜过了任何固定死亡率政策 —— 渔业管理中的标准政策家族。