This paper investigates the design of few-shot exemplars for computer
automation through prompting large language models (LLMs). While previous
prompting approaches focus on self-correction, we find that well-structured
exemplars alone are sufficient for human-level performance. We present Synapse,
an in-context computer control agent demonstrating human-level performance on
the MiniWob++ benchmark. Synapse consists of three main components: 1)
state-conditional decomposition, which divides demonstrations into exemplar
sets based on the agent's need for new environment states, enabling temporal
abstraction; 2) structured prompting, which filters states and reformulates
task descriptions for each set to improve planning correctness; and 3) exemplar
retrieval, which associates incoming tasks with corresponding exemplars in an
exemplar database for multi-task adaptation and generalization. Synapse
overcomes context length limits, reduces errors in multi-step control, and
allows for more exemplars within the context. Importantly, Synapse complements
existing prompting approaches that enhance LLMs' reasoning and planning
abilities. Synapse outperforms previous methods, including behavioral cloning,
reinforcement learning, finetuning, and prompting, with an average success rate
of $98.5\%$ across 63 tasks in MiniWob++. Notably, Synapse relies on exemplars
from only 47 tasks, demonstrating effective generalization to novel tasks. Our
results highlight the potential of in-context learning to advance the
integration of LLMs into practical tool automation.

通过在语言模型中引导少样本范例设计，本文研究了计算机自动化的设计，提出了基于 Sample 和结构化提示的控制代理 Synapse，通过条件分解、结构化提示和样本检索三个部分，能够提高计划的正确性并在 MiniWob++ 基准测试中实现人类级性能，同时也为将 LLMs 融入实际工具自动化提供了新思路。