Transferring policies learned in simulation to the real world is a promising
strategy for acquiring robot skills at scale. However, sim-to-real approaches
typically rely on manual design and tuning of the task reward function as well
as the simulation physics parameters, rendering the process slow and
human-labor intensive. In this paper, we investigate using Large Language
Models (LLMs) to automate and accelerate sim-to-real design. Our LLM-guided
sim-to-real approach, DrEureka, requires only the physics simulation for the
target task and automatically constructs suitable reward functions and domain
randomization distributions to support real-world transfer. We first
demonstrate that our approach can discover sim-to-real configurations that are
competitive with existing human-designed ones on quadruped locomotion and
dexterous manipulation tasks. Then, we showcase that our approach is capable of
solving novel robot tasks, such as quadruped balancing and walking atop a yoga
ball, without iterative manual design.

使用大规模语言模型（LLMs）自动构建适合真实世界转化的奖励函数和域随机化分布的 DrEureka 方法可用于解决四足动作和灵巧操纵等机器人任务，且无需迭代式手动设计。

DrEureka: 语言模型引导的模拟到现实转移

DrEureka: Language Model Guided Sim-To-Real Transfer

Nowadays, the behavior tree is gaining popularity as a representation for
robot tasks due to its modularity and reusability. Designing behavior-tree
tasks manually is time-consuming for robot end-users, thus there is a need for
investigating automatic behavior-tree-based task generation. Prior
behavior-tree-based task generation approaches focus on fixed primitive tasks
and lack generalizability to new task domains. To cope with this issue, we
propose a novel behavior-tree-based task generation approach that utilizes
state-of-the-art large language models. We propose a Phase-Step prompt design
that enables a hierarchical-structured robot task generation and further
integrate it with behavior-tree-embedding-based search to set up the
appropriate prompt. In this way, we enable an automatic and cross-domain
behavior-tree task generation. Our behavior-tree-based task generation approach
does not require a set of pre-defined primitive tasks. End-users only need to
describe an abstract desired task and our proposed approach can swiftly
generate the corresponding behavior tree. A full-process case study is provided
to demonstrate our proposed approach. An ablation study is conducted to
evaluate the effectiveness of our Phase-Step prompts. Assessment on Phase-Step
prompts and the limitation of large language models are presented and
discussed.

提出了一种基于 behavior tree 和大型语言模型的任务生成方法，该方法支持跨领域和自动生成任务，仅需要描述抽象期望任务即可快速生成行为树。

基于行为树和大型语言模型的机器人任务生成

Robot Behavior-Tree-Based Task Generation with Large Language Models

Reinforcement learning is a promising approach for learning control policies
for robot tasks. However, specifying complex tasks (e.g., with multiple
objectives and safety constraints) can be challenging, since the user must
design a reward function that encodes the entire task. Furthermore, the user
often needs to manually shape the reward to ensure convergence of the learning
algorithm. We propose a language for specifying complex control tasks, along
with an algorithm that compiles specifications in our language into a reward
function and automatically performs reward shaping. We implement our approach
in a tool called SPECTRL, and show that it outperforms several state-of-the-art
baselines.

提出一种用于规定复杂控制任务的语言并实现在工具 SPECTRL 中的算法，可以将任务规定编译为奖励函数并自动进行奖励形状。