This scoping survey focuses on our current understanding of the design space
for task-oriented LLM systems and elaborates on definitions and relationships
among the available design parameters. The paper begins by defining a minimal
task-oriented LLM system and exploring the design space of such systems through
a thought experiment contemplating the performance of diverse LLM system
configurations (involving single LLMs, single LLM-based agents, and multiple
LLM-based agent systems) on a complex software development task and
hypothesizes the results. We discuss a pattern in our results and formulate
them into three conjectures. While these conjectures may be partly based on
faulty assumptions, they provide a starting point for future research. The
paper then surveys a select few design parameters: covering and organizing
research in LLM augmentation, prompting techniques, and uncertainty estimation,
and discussing their significance. The paper notes the lack of focus on
computational and energy efficiency in evaluating research in these areas. Our
survey findings provide a basis for developing the concept of linear and
non-linear contexts, which we define and use to enable an agent-centric
projection of prompting techniques providing a lens through which prompting
techniques can be viewed as multi-agent systems. The paper discusses the
implications of this lens, for the cross-pollination of research between LLM
prompting and LLM-based multi-agent systems; and also, for the generation of
synthetic training data based on existing prompting techniques in research. In
all, the scoping survey presents seven conjectures that can help guide future
research efforts.

这项调研重点关注任务导向的 LLM 系统的设计空间，通过思考实验对不同的 LLM 系统配置性能进行探索，并对 LLM 增强、提示技术和不确定性估计等设计参数进行讨论，提出了七个猜想以指导未来的研究努力。