Large Language Model (LLM) -in-the-loop applications have been shown to
effectively interpret the human user's commands, make plans, and operate
external tools/systems accordingly. Still, the operation scope of the LLM agent
is limited to passively following the user, requiring the user to frame his/her
needs with regard to the underlying tools/systems. We note that the potential
of an LLM-Agent User Interface (LAUI) is much greater. A user mostly ignorant
to the underlying tools/systems should be able to work with a LAUI to discover
an emergent workflow. Contrary to the conventional way of designing an
explorable GUI to teach the user a predefined set of ways to use the system, in
the ideal LAUI, the LLM agent is initialized to be proficient with the system,
proactively studies the user and his/her needs, and proposes new interaction
schemes to the user. To illustrate LAUI, we present Flute X GPT, a concrete
example using an LLM agent, a prompt manager, and a flute-tutoring multi-modal
software-hardware system to facilitate the complex, real-time user experience
of learning to play the flute.

使用大规模语言模型（LLM）代理人用户界面（LAUI）能够主动学习用户需求，提出新的交互方案，帮助用户发现新的工作流程，Flute X GPT 是一个具体的例子，利用 LLM 代理人、提示管理器和一个管笛教学的多模式软硬件系统来促进学习吹笛的复杂实时用户体验。