Large Language Models (LLMs) have excelled as high-level semantic planners
for sequential decision-making tasks. However, harnessing them to learn complex
low-level manipulation tasks, such as dexterous pen spinning, remains an open
problem. We bridge this fundamental gap and present Eureka, a human-level
reward design algorithm powered by LLMs. Eureka exploits the remarkable
zero-shot generation, code-writing, and in-context improvement capabilities of
state-of-the-art LLMs, such as GPT-4, to perform evolutionary optimization over
reward code. The resulting rewards can then be used to acquire complex skills
via reinforcement learning. Without any task-specific prompting or pre-defined
reward templates, Eureka generates reward functions that outperform expert
human-engineered rewards. In a diverse suite of 29 open-source RL environments
that include 10 distinct robot morphologies, Eureka outperforms human experts
on 83% of the tasks, leading to an average normalized improvement of 52%. The
generality of Eureka also enables a new gradient-free in-context learning
approach to reinforcement learning from human feedback (RLHF), readily
incorporating human inputs to improve the quality and the safety of the
generated rewards without model updating. Finally, using Eureka rewards in a
curriculum learning setting, we demonstrate for the first time, a simulated
Shadow Hand capable of performing pen spinning tricks, adeptly manipulating a
pen in circles at rapid speed.

Eureka 是一个由 LLMs 支持的人类级奖励设计算法，通过利用 GPT-4 等最先进的 LLMs 的无需提示、代码编写和条件改进能力，进行奖励代码的进化优化，从而生成可用于强化学习的复杂技能获取奖励函数。在一个多样的包含 10 个不同机器人形态的 29 个开源强化学习环境中，Eureka 在 83％的任务上超过了人类专家，在平均标准化改进率为 52％。Eureka 的通用性还使得从人类反馈进行无梯度的上下文学习（RLHF）成为可能，在不进行模型更新的情况下，即可融入人类输入以提高生成奖励的质量和安全性。最后，利用 Eureka 奖励在课程学习设置中演示了模拟的 Shadow Hand 能够熟练进行旋转笔技巧，并以快速速度进行笔的操纵。

Eureka：通过编码大型语言模型实现与人类水平的奖励设计

Eureka: Human-Level Reward Design via Coding Large Language Models

We introduce EUREKA, an ensemble-based approach for performing automatic
euphemism detection. We (1) identify and correct potentially mislabelled rows
in the dataset, (2) curate an expanded corpus called EuphAug, (3) leverage
model representations of Potentially Euphemistic Terms (PETs), and (4) explore
using representations of semantically close sentences to aid in classification.
Using our augmented dataset and kNN-based methods, EUREKA was able to achieve
state-of-the-art results on the public leaderboard of the Euphemism Detection
Shared Task, ranking first with a macro F1 score of 0.881. Our code is
available at this https URL.

EUREKA 是一种基于集成的方法，用于自动检测委婉语，通过对潜在委婉语术语 (PET) 的模型表示和语义相似句子的表示进行分类， EUREKA 能够在公共排行榜上取得最高得分 0.881 的状态