Diffusion-based models for story visualization have shown promise in
generating content-coherent images for storytelling tasks. However, how to
effectively integrate new characters into existing narratives while maintaining
character consistency remains an open problem, particularly with limited data.
Two major limitations hinder the progress: (1) the absence of a suitable
benchmark due to potential character leakage and inconsistent text labeling,
and (2) the challenge of distinguishing between new and old characters, leading
to ambiguous results. To address these challenges, we introduce the NewEpisode
benchmark, comprising refined datasets designed to evaluate generative models'
adaptability in generating new stories with fresh characters using just a
single example story. The refined dataset involves refined text prompts and
eliminates character leakage. Additionally, to mitigate the character confusion
of generated results, we propose EpicEvo, a method that customizes a
diffusion-based visual story generation model with a single story featuring the
new characters seamlessly integrating them into established character dynamics.
EpicEvo introduces a novel adversarial character alignment module to align the
generated images progressively in the diffusive process, with exemplar images
of new characters, while applying knowledge distillation to prevent forgetting
of characters and background details. Our evaluation quantitatively
demonstrates that EpicEvo outperforms existing baselines on the NewEpisode
benchmark, and qualitative studies confirm its superior customization of visual
story generation in diffusion models. In summary, EpicEvo provides an effective
way to incorporate new characters using only one example story, unlocking new
possibilities for applications such as serialized cartoons.

EpicEvo 通过引入新的角色来适应性生成新故事，并将这些角色无缝地融入已有的人物动态中，使用了改良的文本数据集和一种具有对抗角色对齐模块的扩散式视觉故事生成模型，通过定量评估和定性研究证明了其在 Diffusion 模型中定制化视觉故事生成方面的优越性。

演化叙事：新角色定制的基准和方法与扩散模型

Evolving Storytelling: Benchmarks and Methods for New Character  Customization with Diffusion Models

Diffusion models developed on top of powerful text-to-image generation models
like Stable Diffusion achieve remarkable success in visual story generation.
However, the best-performing approach considers historically generated results
as flattened memory cells, ignoring the fact that not all preceding images
contribute equally to the generation of the characters and scenes at the
current stage. To address this, we present a simple method that improves the
leading system with adaptive context modeling, which is not only incorporated
in the encoder but also adopted as additional guidance in the sampling stage to
boost the global consistency of the generated story. We evaluate our model on
PororoSV and FlintstonesSV datasets and show that our approach achieves
state-of-the-art FID scores on both story visualization and continuation
scenarios. We conduct detailed model analysis and show that our model excels at
generating semantically consistent images for stories.

提出了一种在编码器和采样阶段均采用自适应上下文建模的方法来改进文本到图像生成模型，实现了对视觉化故事生成的全局一致性的提升，取得了 PororoSV 和 FlintstonesSV 数据集上最佳 FID 分数和生成语义一致故事图像的优异性能。