Large vision-language models are steadily gaining personalization
capabilities at the cost of fine-tuning or data augmentation. We present two
models for image generation using model-agnostic learning that align semantic
priors with generative capabilities. RLDF, or Reinforcement Learning from
Diffusion Feedback, is a singular approach for visual imitation through
prior-preserving reward function guidance. This employs Q-learning (with
standard Q*) for generation and follows a semantic-rewarded trajectory for
image search through finite encoding-tailored actions. The second proposed
method, noisy diffusion gradient, is optimization driven. At the root of both
methods is a special CFG encoding that we propose for continual semantic
guidance. Using only a single input image and no text input, RLDF generates
high-quality images over varied domains including retail, sports and
agriculture showcasing class-consistency and strong visual diversity. Project
website is available at this https URL

大型视觉 - 语言模型通过非微调或数据增强实现个性化能力增强。本文提出了两种使用模型无关学习进行图像生成的模型，通过将语义先验与生成能力进行对齐。其中一种方法是 RLDF（Reinforcement Learning from Diffusion Feedback），通过保持先前的奖励函数进行视觉模仿。另一种方法是噪声扩散梯度优化方法。这些方法的核心是我们提出的连续语义引导的特殊 CFG 编码。RLDF 仅使用单张输入图像且无文本输入，在不同领域包括零售、体育和农业生成了具有类一致性和强大视觉多样性的高质量图像。项目网站可在此 https URL 找到。

从扩散反馈中强化学习：图像搜索的 Q*

Reinforcement Learning from Diffusion Feedback: Q* for Image Search

Imitation learning trains control policies by mimicking pre-recorded expert
demonstrations. In partially observable settings, imitation policies must rely
on observation histories, but many seemingly paradoxical results show better
performance for policies that only access the most recent observation. Recent
solutions ranging from causal graph learning to deep information bottlenecks
have shown promising results, but failed to scale to realistic settings such as
visual imitation. We propose a solution that outperforms these prior approaches
by upweighting demonstration keyframes corresponding to expert action
changepoints. This simple approach easily scales to complex visual imitation
settings. Our experimental results demonstrate consistent performance
improvements over all baselines on image-based Gym MuJoCo continuous control
tasks. Finally, on the CARLA photorealistic vision-based urban driving
simulator, we resolve a long-standing issue in behavioral cloning for driving
by demonstrating effective imitation from observation histories. Supplementary
materials and code at: https://tinyurl.com/imitation-keyframes.

该研究提出了一种通过加强模仿的关键帧来改进模仿学习的方法，以在视觉模仿等现实场景中实现更好的性能表现，并在基于图像和基于视觉的控制任务中进行了验证。