The rise of Foundation Models (FMs) like Large Language Models (LLMs) is revolutionizing software development. Despite the impressive prototypes, transforming FMware into production-ready products demands complex engineering across various domains. A critical but overlooked aspect is performance engineering, which aims at ensuring FMware meets performance goals such as throughput and latency to avoid user dissatisfaction and financial loss. Often, performance considerations are an afterthought, leading to costly optimization efforts post-deployment. FMware's high computational resource demands highlight the need for efficient hardware use. Continuous performance engineering is essential to prevent degradation. This paper highlights the significance of Software Performance Engineering (SPE) in FMware, identifying four key challenges: cognitive architecture design, communication protocols, tuning and optimization, and deployment. These challenges are based on literature surveys and experiences from developing an in-house FMware system. We discuss problems, current practices, and innovative paths for the software engineering community.

本研究解决了基础模型驱动软件（FMware）在性能工程方面的不足，强调了为确保软件达到高性能需求而进行的系统性工程的重要性。作者识别了认知架构设计、通信协议、调优与优化、以及部署等四个关键挑战，并提出了针对软件工程社区的创新解决路径，旨在改善FMware的生产准备性。通过这些努力，研究预计将对用户满意度和运营效率产生显著影响。

基础模型驱动软件的软件性能工程（FMware）