We present SpaceAgents-1, a system for learning human and multi-robot
collaboration (HMRC) strategies under microgravity conditions. Future space
exploration requires humans to work together with robots. However, acquiring
proficient robot skills and adept collaboration under microgravity conditions
poses significant challenges within ground laboratories. To address this issue,
we develop a microgravity simulation environment and present three typical
configurations of intra-cabin robots. We propose a hierarchical heterogeneous
multi-agent collaboration architecture: guided by foundation models, a
Decision-Making Agent serves as a task planner for human-robot collaboration,
while individual Skill-Expert Agents manage the embodied control of robots.
This mechanism empowers the SpaceAgents-1 system to execute a range of
intricate long-horizon HMRC tasks.

SpaceAgents-1 系统在微重力环境下学习人机和多机器人协作策略，通过开发微重力模拟环境和引入分层异构多机器人协作架构，解决了地面实验室中获得熟练机器人技能和在微重力条件下进行高效协作的难题。

我们选择去太空：人与多机器人在失重状态下的代理驱动协作

We Choose to Go to Space: Agent-driven Human and Multi-Robot  Collaboration in Microgravity

Recent works have shown that Large Language Models (LLMs) can be applied to
ground natural language to a wide variety of robot skills. However, in
practice, learning multi-task, language-conditioned robotic skills typically
requires large-scale data collection and frequent human intervention to reset
the environment or help correcting the current policies. In this work, we
propose a novel approach to efficiently learn general-purpose
language-conditioned robot skills from unstructured, offline and reset-free
data in the real world by exploiting a self-supervised visuo-lingual affordance
model, which requires annotating as little as 1% of the total data with
language. We evaluate our method in extensive experiments both in simulated and
real-world robotic tasks, achieving state-of-the-art performance on the
challenging CALVIN benchmark and learning over 25 distinct visuomotor
manipulation tasks with a single policy in the real world. We find that when
paired with LLMs to break down abstract natural language instructions into
subgoals via few-shot prompting, our method is capable of completing
long-horizon, multi-tier tasks in the real world, while requiring an order of
magnitude less data than previous approaches. Code and videos are available at
this http URL

本文提出了一种基于视觉语言感知模型的新方法，可以在真实世界中从非结构化、离线、无需重置的数据中高效地学习通用的、以语言为条件的机器人技能，实验表明该方法配合 LLMS 可以在真实世界中完成长期、多层次的任务，并比以往方法少使用一个数量级的数据。

利用视觉可供性在非结构化数据上进行语言基础建设

Grounding Language with Visual Affordances over Unstructured Data

Automated task planning algorithms have been developed to help robots
complete complex tasks that require multiple actions. Most of those algorithms
have been developed for "closed worlds" assuming complete world knowledge is
provided. However, the real world is generally open, and the robots frequently
encounter unforeseen situations that can potentially break the planner's
completeness. This paper introduces a novel algorithm (COWP) for open-world
task planning and situation handling that dynamically augments the robot's
action knowledge with task-oriented common sense. In particular, common sense
is extracted from Large Language Models based on the current task at hand and
robot skills. For systematic evaluations, we collected a dataset that includes
561 execution-time situations in a dining domain, where each situation
corresponds to a state instance of a robot being potentially unable to complete
a task using a solution that normally works. Experimental results show that our
approach significantly outperforms competitive baselines from the literature in
the success rate of service tasks. Additionally, we have demonstrated COWP
using a mobile manipulator. Supplementary materials are available at:
this https URL

介绍了一种新的算法 (COWP)，用于基于任务和机器人技能从语言模型中提取常识，实现在开放世界中完成任务规划和情境处理，实验结果表明其在服务任务成功率方面显著优于现有文献中的竞争基线算法。