Personalized text generation is an emerging research area that has attracted
much attention in recent years. Most studies in this direction focus on a
particular domain by designing bespoke features or models. In this work, we
propose a general approach for personalized text generation using large
language models (LLMs). Inspired by the practice of writing education, we
develop a multistage and multitask framework to teach LLMs for personalized
generation. In writing instruction, the task of writing from sources is often
decomposed into multiple steps that involve finding, evaluating, summarizing,
synthesizing, and integrating information. Analogously, our approach to
personalized text generation consists of multiple stages: retrieval, ranking,
summarization, synthesis, and generation. In addition, we introduce a multitask
setting that helps the model improve its generation ability further, which is
inspired by the observation in education that a student's reading proficiency
and writing ability are often correlated. We evaluate our approach on three
public datasets, each of which covers a different and representative domain.
Our results show significant improvements over a variety of baselines.

个性化文本生成是一个新兴的研究领域，通过使用大型语言模型 (LLMs)，我们提出了一种通用的个性化文本生成方法。借鉴写作教育的实践，我们开发了一个多阶段和多任务的框架来教授 LLMs 进行个性化生成。我们的方法包括检索、排名、总结、综合和生成多个阶段，并引入了一个多任务设置来进一步提高模型的生成能力。我们在三个涵盖不同代表性领域的公开数据集上评估了我们的方法，结果显示相对于各种基线模型，我们取得了显著的改进。

教导 LLM 个性化 —— 受到写作教育启发的方法

Teach LLMs to Personalize -- An Approach inspired by Writing Education

This technical report presents the modeling approaches used in our submission
to the ICML Expressive Vocalizations Workshop & Competition multitask track
(ExVo-MultiTask). We first applied image classification models of various sizes
on mel-spectrogram representations of the vocal bursts, as is standard in sound
event detection literature. Results from these models show an increase of
21.24% over the baseline system with respect to the harmonic mean of the task
metrics, and comprise our team's main submission to the MultiTask track. We
then sought to characterize the headroom in the MultiTask track by applying a
large pre-trained Conformer model that previously achieved state-of-the-art
results on paralinguistic tasks like speech emotion recognition and mask
detection. We additionally investigated the relationship between the sub-tasks
of emotional expression, country of origin, and age prediction, and discovered
that the best performing models are trained as single-task models, questioning
whether the problem truly benefits from a multitask setting.

本文介绍了我们在 ICML 表达性声学竞赛中使用的建模方法，其中采用了多种卷积神经网络模型，并且发现 Conformer 模型在该任务中存在潜力，并提出在情感表达、年龄预测和来源国家等子任务方面，使用单任务模型效果更好，质疑了该问题是否真正从多任务的设置中受益。