Style is an integral component of text that expresses a diverse set of
information, including interpersonal dynamics (e.g. formality) and the author's
emotions or attitudes (e.g. disgust). Humans often employ multiple styles
simultaneously. An open question is how large language models can be explicitly
controlled so that they weave together target styles when generating text: for
example, to produce text that is both negative and non-toxic. Previous work
investigates the controlled generation of a single style, or else controlled
generation of a style and other attributes. In this paper, we expand this into
controlling multiple styles simultaneously. Specifically, we investigate
various formulations of multiple style rewards for a reinforcement learning
(RL) approach to controlled multi-style generation. These reward formulations
include calibrated outputs from discriminators and dynamic weighting by
discriminator gradient magnitudes. We find that dynamic weighting generally
outperforms static weighting approaches, and we explore its effectiveness in 2-
and 3-style control, even compared to strong baselines like plug-and-play
model. All code and data for RL pipelines with multiple style attributes will
be publicly available.

通过强化学习方法控制多种风格的生成，使用动态权重方法优于静态权重方法，并在 2 个和 3 个风格控制方面进行了实证探索。

基于动态多重奖励权重的多样式可控生成的强化学习

Reinforcement Learning with Dynamic Multi-Reward Weighting for  Multi-Style Controllable Generation

Stylized visual captioning aims to generate image or video descriptions with
specific styles, making them more attractive and emotionally appropriate. One
major challenge with this task is the lack of paired stylized captions for
visual content, so most existing works focus on unsupervised methods that do
not rely on parallel datasets. However, these approaches still require training
with sufficient examples that have style labels, and the generated captions are
limited to predefined styles. To address these limitations, we explore the
problem of Few-Shot Stylized Visual Captioning, which aims to generate captions
in any desired style, using only a few examples as guidance during inference,
without requiring further training. We propose a framework called FS-StyleCap
for this task, which utilizes a conditional encoder-decoder language model and
a visual projection module. Our two-step training scheme proceeds as follows:
first, we train a style extractor to generate style representations on an
unlabeled text-only corpus. Then, we freeze the extractor and enable our
decoder to generate stylized descriptions based on the extracted style vector
and projected visual content vectors. During inference, our model can generate
desired stylized captions by deriving the style representation from
user-supplied examples. Our automatic evaluation results for few-shot
sentimental visual captioning outperform state-of-the-art approaches and are
comparable to models that are fully trained on labeled style corpora. Human
evaluations further confirm our model s ability to handle multiple styles.

FS-StyleCap 是一个用于 Few-Shot Stylized Visual Captioning 的框架，通过训练样本生成相关风格的标题描述，其在自动评估中的结果优于现有方法，并且在处理多种风格方面具有能力。