Numerous recent techniques for text style transfer characterize their
approaches as variants of reinforcement learning and preference optimization.
In this work, we consider the relationship between these approaches and a class
of optimization approaches developed primarily for (non-neural) statistical
machine translation, formerly known as 'tuning'. Inspired by these techniques
from the past, we improve upon established preference optimization approaches,
incorporating multiple iterations of exploration and optimization, and choosing
contrastive examples by following a 'hope' vs 'fear' sampling strategy.
Cognizant of the difference between machine translation and style transfer,
however, we further tailor our framework with a new pseudo-parallel generation
method and a dynamic weighted reward aggregation method to tackle the lack of
parallel data and the need for a multi-objective reward. We evaluate our model
on two commonly used text style transfer datasets. Through automatic and human
evaluation results we show the effectiveness and the superiority of our model
compared to state-of-the-art baselines.

通过引入探索和优化的多次迭代和希望与恐惧采样策略，以及伪平行生成方法和动态加权奖励聚合方法来改进偏好优化方法，本文在两个常用的文本风格转换数据集上评估模型，通过自动化和人工评估结果展示了我们模型相对于最先进基准的有效性和优越性。