AlayaDB is a cutting-edge vector database system natively architected for efficient and effective long-context inference for Large Language Models (LLMs) at AlayaDB AI. Specifically, it decouples the KV cache and attention computation from the LLM inference systems, and encapsulates them into a novel vector database system. For the Model as a Service providers (MaaS), AlayaDB consumes fewer hardware resources and offers higher generation quality for various workloads with different kinds of Service Level Objectives (SLOs), when comparing with the existing alternative solutions (e.g., KV cache disaggregation, retrieval-based sparse attention). The crux of AlayaDB is that it abstracts the attention computation and cache management for LLM inference into a query processing procedure, and optimizes the performance via a native query optimizer. In this work, we demonstrate the effectiveness of AlayaDB via (i) three use cases from our industry partners, and (ii) extensive experimental results on LLM inference benchmarks.

本研究针对现有大型语言模型推理系统在长上下文处理中的低效问题，提出了AlayaDB，一个新型向量数据库系统。该系统通过将KV缓存和注意力计算与推理过程解耦，从而减少硬件资源消耗，并在多种服务级别目标下提供更高的生成质量，展现了良好的实用性和性能优化。

AlayaDB：高效且有效的长上下文大型语言模型推理的数据基础