Modern AI models provide the key to a long-standing dream: processing
analytical queries about almost any kind of data. Until recently, it was
difficult and expensive to extract facts from company documents, data from
scientific papers, or insights from image and video corpora. Today's models can
accomplish these tasks with high accuracy. However, a programmer who wants to
answer a substantive AI-powered query must orchestrate large numbers of models,
prompts, and data operations. For even a single query, the programmer has to
make a vast number of decisions such as the choice of model, the right
inference method, the most cost-effective inference hardware, the ideal prompt
design, and so on. The optimal set of decisions can change as the query changes
and as the rapidly-evolving technical landscape shifts. In this paper we
present Palimpzest, a system that enables anyone to process AI-powered
analytical queries simply by defining them in a declarative language. The
system uses its cost optimization framework -- which explores the search space
of AI models, prompting techniques, and related foundation model optimizations
-- to implement the query with the best trade-offs between runtime, financial
cost, and output data quality. We describe the workload of AI-powered analytics
tasks, the optimization methods that Palimpzest uses, and the prototype system
itself. We evaluate Palimpzest on tasks in Legal Discovery, Real Estate Search,
and Medical Schema Matching. We show that even our simple prototype offers a
range of appealing plans, including one that is 3.3x faster, 2.9x cheaper, and
offers better data quality than the baseline method. With parallelism enabled,
Palimpzest can produce plans with up to a 90.3x speedup at 9.1x lower cost
relative to a single-threaded GPT-4 baseline, while obtaining an F1-score
within 83.5% of the baseline. These require no additional work by the user.

Palimpzest 是一个系统，它通过在一个声明性语言中定义查询来使任何人能够处理基于人工智能的分析查询。系统使用其成本优化框架来实现查询，在运行时间、财务成本和输出数据质量之间实现最佳权衡。该系统在法律发现、房地产搜索和医疗模式匹配等任务上展示了各种吸引人的计划，包括比基准方法更快 3.3 倍、更便宜 2.9 倍，并提供更好的数据质量的计划。与单线程的 GPT-4 基准相比，通过启用并行性，Palimpzest 可以以最多 90.3 倍的加速度、9.1 倍较低的成本获得 83.5% 的 F1 分数，而且不需要用户额外的工作。