This paper studies retrieval-augmented approaches for personalizing large
language models (LLMs), which potentially have a substantial impact on various
applications and domains. We propose the first attempt to optimize the
retrieval models that deliver a limited number of personal documents to large
language models for the purpose of personalized generation. We develop two
optimization algorithms that solicit feedback from the downstream personalized
generation tasks for retrieval optimization--one based on reinforcement
learning whose reward function is defined using any arbitrary metric for
personalized generation and another based on knowledge distillation from the
downstream LLM to the retrieval model. This paper also introduces a pre- and
post-generation retriever selection model that decides what retriever to choose
for each LLM input. Extensive experiments on diverse tasks from the language
model personalization (LaMP) benchmark reveal statistically significant
improvements in six out of seven datasets.

本研究论文探讨了扩展的检索方法用于个性化大型语言模型，通过两种优化算法从下游任务获取反馈进行检索优化，并引入了一个预生成和后生成的检索模型来决定每个语言模型输入应选择哪个检索器。在多个任务中进行了大量实验，并获得了显著的统计结果。