Animals and robots exist in a physical world and must coordinate their bodies
to achieve behavioral objectives. With recent developments in deep
reinforcement learning, it is now possible for scientists and engineers to
obtain sensorimotor strategies (policies) for specific tasks using physically
simulated bodies and environments. However, the utility of these methods goes
beyond the constraints of a specific task; they offer an exciting framework for
understanding the organization of an animal sensorimotor system in connection
to its morphology and physical interaction with the environment, as well as for
deriving general design rules for sensing and actuation in robotic systems.
Algorithms and code implementing both learning agents and environments are
increasingly available, but the basic assumptions and choices that go into the
formulation of an embodied feedback control problem using deep reinforcement
learning may not be immediately apparent. Here, we present a concise exposition
of the mathematical and algorithmic aspects of model-free reinforcement
learning, specifically through the use of \textit{actor-critic} methods, as a
tool for investigating the feedback control underlying animal and robotic
behavior.

动物和机器人存在于物理世界中，并必须协调其身体以实现行为目标。最近在深度强化学习方面的发展使得科学家和工程师可以利用物理模拟的身体和环境来获得特定任务的感觉运动策略（策略）。然而，这些方法的效用超出了特定任务的限制；它们为理解动物感觉运动系统与其形态和与环境的物理相互作用的组织提供了一个令人兴奋的框架，同时也可为机器人系统中感知和执行的一般设计规则的推导提供支持。通过使用深度强化学习中的	extit {演员 - 评论家} 方法，我们在这里提出数学和算法方面的简洁阐述作为研究动物和机器人行为背后的反馈控制的工具。