We present a pixel-based deep active inference algorithm (PixelAI) inspired
by human body perception and action. Our algorithm combines the free-energy
principle from neuroscience, rooted in variational inference, with deep
convolutional decoders to scale the algorithm to directly deal with raw visual
input and provide online adaptive inference. Our approach is validated by
studying body perception and action in a simulated and a real Nao robot.
Results show that our approach allows the robot to perform 1) dynamical body
estimation of its arm using only monocular camera images and 2) autonomous
reaching to "imagined" arm poses in the visual space. This suggests that robot
and human body perception and action can be efficiently solved by viewing both
as an active inference problem guided by ongoing sensory input.

我们提出了一种基于像素的深度主动推理算法 (PixelAI)，它受到人类身体感知和动作的启发。这种算法将来自神经科学的自由能原理与深度卷积解码器相结合，以直接处理原始视觉输入并提供在线自适应推理。我们的方法通过研究模拟和真实的 Nao 机器人中的身体感知和动作进行验证。结果显示，我们的方法允许机器人通过单眼摄像头图像动态地估计其臂的位置，并自主地朝着 “想象” 的外部位置进行运动，这表明机器人和人类的身体感知和行动可以通过将二者视为由不断的感官输入引导的主动推理问题来有效地解决。

基于像素的端到端深度主动推断在身体感知和动作中的应用

End-to-End Pixel-Based Deep Active Inference for Body Perception and  Action

Providing artificial agents with the same computational models of biological
systems is a way to understand how intelligent behaviours may emerge. We
present an active inference body perception and action model working for the
first time in a humanoid robot. The model relies on the free energy principle
proposed for the brain, where both perception and action goal is to minimise
the prediction error through gradient descent on the variational free energy
bound. The body state (latent variable) is inferred by minimising the
difference between the observed (visual and proprioceptive) sensor values and
the predicted ones. Simultaneously, the action makes sensory data sampling to
better correspond to the prediction made by the inner model. We formalised and
implemented the algorithm on the iCub robot and tested in 2D and 3D visual
spaces for online adaptation to visual changes, sensory noise and discrepancies
between the model and the real robot. We also compared our approach with
classical inverse kinematics in a reaching task, analysing the suitability of
such a neuroscience-inspired approach for real-world interaction. The algorithm
gave the robot adaptive body perception and upper body reaching with head
object tracking (toddler-like), and was able to incorporate visual features
online (in a closed-loop manner) without increasing the computational
complexity. Moreover, our model predicted involuntary actions in the presence
of sensorimotor conflicts showing the path for a potential proof of active
inference in humans.

通过在仿生系统中应用主动推理身体感知和行为模型，能够在机器人上实现适应性身体感知和动作，并分析该方法适用于现实世界的交互。