This paper highlights the importance of personalization in the current state
of natural language understanding and generation and introduces the LaMP
benchmark -- a novel benchmark for training and evaluating language models for
producing personalized outputs. LaMP offers a comprehensive evaluation
framework with diverse language tasks and multiple entries for each user
profile. It consists of seven personalized tasks, spanning three classification
and four text generation tasks. We also propose a retrieval augmentation
approach that retrieves personalized items from user profiles to construct
personalized prompts for large language models. Our baseline zero-shot and
fine-tuned model results indicate that LMs utilizing profile augmentation
outperform their counterparts that do not factor in profile information.

本文介绍了个性化对于自然语言理解和生成的重要性，并引入了 LaMP 基准测试 - 一种用于训练和评估用于生成个性化输出的语言模型的新基准测试。LaMP 提供了一个包含多样化语言任务和每个用户配置文件的多个条目的全面评估框架。它由七个个性化任务组成，横跨三个分类任务和四个文本生成任务。我们还提出了一种检索增强方法，该方法从用户配置文件中检索个性化项，以构造大型语言模型的个性化提示。我们的基线零 - shot 和微调模型结果表明，利用配置文件增强的 LM 优于不考虑配置文件信息的其同类。