Large Language Models (LLMs) excel at a wide range of tasks, but adapting them to new data, particularly for personalized applications, poses significant challenges due to resource and computational constraints. Existing methods either rely on exposing fresh data to the model through the prompt, which is limited by context size and computationally expensive at inference time, or fine-tuning, which incurs substantial training and update costs. In this paper, we introduce ComMer - Compress and Merge - a novel framework that efficiently personalizes LLMs by compressing users' documents into compact representations, which are then merged and fed into a frozen LLM. We evaluate ComMer on two types of personalization tasks - personalized skill learning, using the tweet paraphrasing dataset and the personalized news headline generation dataset from the LaMP benchmark, and knowledge-intensive, using the PerLTQA dataset. Our experiments demonstrate that in constrained inference budget scenarios ComMer achieves superior quality in skill learning tasks, while highlighting limitations in knowledge-intensive settings due to the loss of detailed information. These results offer insights into trade-offs and potential optimizations in multi-document compression for personalization.

本研究针对大语言模型在个性化应用中的适应挑战，提出了ComMer框架，通过将用户文档压缩为紧凑表示，并将其合并后输入到固定的LLM中，从而实现高效个性化。实验表明，在受限的推理预算场景下，ComMer在技能学习任务中表现优异，尽管在知识密集型任务中由于信息损失而存在局限性，提示了多文档压缩在个性化中的权衡与优化潜力。

ComMer：个性化用户数据压缩与合并框架