As large language models (LLMs) improve their capabilities in handling complex tasks, the issues of computational cost and efficiency due to long prompts are becoming increasingly prominent. To accelerate model inference and reduce costs, we propose an innovative prompt compression framework called LanguaShrink. Inspired by the observation that LLM performance depends on the density and position of key information in the input prompts, LanguaShrink leverages psycholinguistic principles and the Ebbinghaus memory curve to achieve task-agnostic prompt compression. This effectively reduces prompt length while preserving essential information. We referred to the training method of OpenChat.The framework introduces part-of-speech priority compression and data distillation techniques, using smaller models to learn compression targets and employing a KL-regularized reinforcement learning strategy for training.\cite{wang2023openchat} Additionally, we adopt a chunk-based compression algorithm to achieve adjustable compression rates. We evaluate our method on multiple datasets, including LongBench, ZeroScrolls, Arxiv Articles, and a newly constructed novel test set. Experimental results show that LanguaShrink maintains semantic similarity while achieving up to 26 times compression. Compared to existing prompt compression methods, LanguaShrink improves end-to-end latency by 1.43 times.

本研究针对大型语言模型在处理长提示时的计算成本与效率问题，提出了一种名为LanguaShrink的创新性提示压缩框架。该框架利用心理语言学原理和艾宾浩斯记忆曲线，实现任务无关的提示压缩，显著减少提示长度的同时保留关键信息，实验结果显示在保持语义相似性的情况下，压缩比可达26倍，并且比现有方法提高了1.43倍的端到端延迟性能。

LanguaShrink：利用心理语言学减少令牌开销