The language-conditioned robotic manipulation aims to transfer natural
language instructions into executable actions, from simple pick-and-place to
tasks requiring intent recognition and visual reasoning. Inspired by the dual
process theory in cognitive science, which suggests two parallel systems of
fast and slow thinking in human decision-making, we introduce Robotics with
Fast and Slow Thinking (RFST), a framework that mimics human cognitive
architecture to classify tasks and makes decisions on two systems based on
instruction types. Our RFST consists of two key components: 1) an instruction
discriminator to determine which system should be activated based on the
current user instruction, and 2) a slow-thinking system that is comprised of a
fine-tuned vision language model aligned with the policy networks, which allows
the robot to recognize user intention or perform reasoning tasks. To assess our
methodology, we built a dataset featuring real-world trajectories, capturing
actions ranging from spontaneous impulses to tasks requiring deliberate
contemplation. Our results, both in simulation and real-world scenarios,
confirm that our approach adeptly manages intricate tasks that demand intent
recognition and reasoning. The project is available at
this https URL

通过模仿人类认知结构，我们引入了一种名为 RFST 的框架，该框架利用语言指令对任务进行分类并基于指令类型做出决策，包含指令鉴别器和一个与策略网络相匹配的精调视觉语言模型，可以识别用户意图或执行推理任务，我们的实验结果验证了我们的方法在需要意图识别和推理的复杂任务中的出色表现。