In order to build artificial intelligence systems that can perceive and
reason with human behavior in the real world, we must first design models that
conduct complex spatio-temporal reasoning over motion sequences. Moving towards
this goal, we propose the HumanMotionQA task to evaluate complex, multi-step
reasoning abilities of models on long-form human motion sequences. We generate
a dataset of question-answer pairs that require detecting motor cues in small
portions of motion sequences, reasoning temporally about when events occur, and
querying specific motion attributes. In addition, we propose NSPose, a
neuro-symbolic method for this task that uses symbolic reasoning and a modular
design to ground motion through learning motion concepts, attribute neural
operators, and temporal relations. We demonstrate the suitability of NSPose for
the HumanMotionQA task, outperforming all baseline methods.

我们提出了一个名为 HumanMotionQA 的任务来评估模型在长形式人体运动序列上的复杂多步推理能力，同时提出了一种名为 NSPose 的神经符号方法。该方法使用符号推理和模块化设计通过学习运动概念、属性神经算子和时间关系来地面运动。我们演示了 NSPose 在 HumanMotionQA 任务上的适用性，优于所有基线方法。