We introduce WorkBench: a benchmark dataset for evaluating agents' ability to
execute tasks in a workplace setting. WorkBench contains a sandbox environment
with five databases, 26 tools, and 690 tasks. These tasks represent common
business activities, such as sending emails and scheduling meetings. The tasks
in WorkBench are challenging as they require planning, tool selection, and
often multiple actions. If a task has been successfully executed, one (or more)
of the database values may change. The correct outcome for each task is unique
and unambiguous, which allows for robust, automated evaluation. We call this
key contribution outcome-centric evaluation. We evaluate five existing ReAct
agents on WorkBench, finding they successfully complete as few as 3% of tasks
(Llama2-70B), and just 43% for the best-performing (GPT-4). We further find
that agents' errors can result in the wrong action being taken, such as an
email being sent to the wrong person. WorkBench reveals weaknesses in agents'
ability to undertake common business activities, raising questions about their
use in high-stakes workplace settings. WorkBench is publicly available as a
free resource at this https URL

WorkBench 是一个用于评估工作场所环境中代理程序执行任务能力的基准数据集，其中包含五个数据库、26 个工具和 690 个任务。通过对五种现有代理程序在 WorkBench 上的评估，发现它们成功完成任务的比例在 3%（Llama2-70B）到 43%（GPT-4）之间，同时也揭示了代理程序在常见商务活动中的能力弱点，进而引发对其在高风险工作环境中使用的质疑。