It is doubtful that animals have perfect inverse models of their limbs (e.g., what muscle contraction must be applied to every joint to reach a particular location in space). However, in robot control, moving an arm's end-effector to a target position or along a target trajectory requires accurate forward and inverse models. Here we show that by learning the transition (forward) model from interaction, we can use it to drive the learning of an amortized policy. Hence, we revisit policy optimization in relation to the deep active inference framework and describe a modular neural network architecture that simultaneously learns the system dynamics from prediction errors and the stochastic policy that generates suitable continuous control commands to reach a desired reference position. We evaluated the model by comparing it against the baseline of a linear quadratic regulator, and conclude with additional steps to take toward human-like motor control.

通过学习过渡模型，我们可以使用它来驱动摊销政策的学习。因此，我们重访了政策优化并描述了一个模块化神经网络架构，该架构从预测错误中同时学习了系统动力学和生成适当连续控制命令的随机策略，并通过与线性二次调节器的基线进行比较来评估模型。

通过转移模型学习连续控制策略