Large language models (LLMs) require lengthy prompts as the input context to
produce output aligned with user intentions, a process that incurs extra costs
during inference. In this paper, we propose the Gist COnditioned deCOding
(Gist-COCO) model, introducing a novel method for compressing prompts which
also can assist the prompt interpretation and engineering. Gist-COCO employs an
encoder-decoder based language model and then incorporates an additional
encoder as a plugin module to compress prompts with inputs using gist tokens.
It finetunes the compression plugin module and uses the representations of gist
tokens to emulate the raw prompts in the vanilla language model. By verbalizing
the representations of gist tokens into gist prompts, the compression ability
of Gist-COCO can be generalized to different LLMs with high compression rates.
Our experiments demonstrate that Gist-COCO outperforms previous prompt
compression models in both passage and instruction compression tasks. Further
analysis on gist verbalization results suggests that our gist prompts serve
different functions in aiding language models. They may directly provide
potential answers, generate the chain-of-thought, or simply repeat the inputs.
All data and codes are available at this https URL .

该研究提出了 Gist COnditioned deCOding (Gist-COCO) 模型，利用编码器和解码器构建语言模型，并使用附加的编码器插件模块进行输入压缩，将表示的要点标记转化为要点提示，以高压缩率优于以往的要点压缩模型，辅助大型语言模型在不同任务中的表现。