Large Language Models (LLMs) have showcased remarkable capabilities
surpassing conventional NLP challenges, creating opportunities for use in
production use cases. Towards this goal, there is a notable shift to building
compound AI systems, wherein LLMs are integrated into an expansive software
infrastructure with many components like models, retrievers, databases and
tools. In this paper, we introduce a blueprint architecture for compound AI
systems to operate in enterprise settings cost-effectively and feasibly. Our
proposed architecture aims for seamless integration with existing compute and
data infrastructure, with ``stream'' serving as the key orchestration concept
to coordinate data and instructions among agents and other components. Task and
data planners, respectively, break down, map, and optimize tasks and data to
available agents and data sources defined in respective registries, given
production constraints such as accuracy and latency.

大型语言模型已经展示出了超越传统自然语言处理挑战的卓越能力，为生产使用案例创造了机会。为了实现这一目标，出现了构建复合人工智能系统的显著转变，在其中将大型语言模型与包括模型、检索器、数据库和工具在内的广泛软件基础设施集成。本文介绍了一种针对复合人工智能系统在企业环境中进行成本效益和可行性操作的蓝图架构。我们提出的架构旨在与现有的计算和数据基础设施进行无缝集成，通过 “流” 作为关键的编排概念，在代理和其他组件之间协调数据和指令。任务和数据计划分别将任务和数据分解、映射并优化分配给在各自注册表中定义的可用代理和数据源，考虑生产约束条件，如准确度和延迟。

面向企业的复合人工智能系统的蓝图架构

A Blueprint Architecture of Compound AI Systems for Enterprise

Although the rise of Large Language Models (LLMs) in enterprise settings
brings new opportunities and capabilities, it also brings challenges, such as
the risk of generating inappropriate, biased, or misleading content that
violates regulations and can have legal concerns. To alleviate this, we present
"LLMGuard", a tool that monitors user interactions with an LLM application and
flags content against specific behaviours or conversation topics. To do this
robustly, LLMGuard employs an ensemble of detectors.

通过使用一组检测器，我们提出了 “LLMGuard”，这是一个监视用户与 LLM 应用程序交互并对内容进行标记的工具，以应对大型语言模型在企业环境中带来的新机遇和挑战。