We formalize the problem of prompt compression for large language models (LLMs) and present a framework to unify token-level prompt compression methods which create hard prompts for black-box models. We derive the distortion-rate function for this setup as a linear program, and provide an efficient algorithm to compute this fundamental limit via the dual of the linear program. Using the distortion-rate function as the baseline, we study the performance of existing compression schemes on a synthetic dataset consisting of prompts generated from a Markov chain, natural language queries, and their respective answers. Our empirical analysis demonstrates the criticality of query-aware prompt compression, where the compressor has knowledge of the downstream task/query for the black-box LLM. We show that there is a large gap between the performance of current prompt compression methods and the optimal strategy, and propose a query-aware, variable-rate adaptation of a prior work to close the gap. We extend our experiments to a small natural language dataset to further confirm our findings on our synthetic dataset.

本研究针对大型语言模型的提示压缩问题进行了形式化，并提出了一个框架来统一生成硬提示的令牌级压缩方法。我们导出了该设置的失真-率函数，并证明了现有压缩方案与最优策略之间存在巨大差距，提出了一种查询感知、可变率的适应方法，以缩小这一差距。

提示压缩的基本限制：用于黑箱语言模型的率-失真框架