Text-to-image models such as stable diffusion have opened a plethora of
opportunities for generating art. Recent literature has surveyed the use of
text-to-image models for enhancing the work of many creative artists. Many
e-commerce platforms employ a manual process to generate the banners, which is
time-consuming and has limitations of scalability. In this work, we demonstrate
the use of text-to-image models for generating personalized web banners with
dynamic content for online shoppers based on their interactions. The novelty in
this approach lies in converting users' interaction data to meaningful prompts
without human intervention. To this end, we utilize a large language model
(LLM) to systematically extract a tuple of attributes from item
meta-information. The attributes are then passed to a text-to-image model via
prompt engineering to generate images for the banner. Our results show that the
proposed approach can create high-quality personalized banners for users.

利用文本与图像模型生成个性化网页横幅，基于用户的互动行为生成动态内容的横幅，通过大型语言模型系统地提取项目元信息的属性，并通过提示工程将属性传递给文本与图像模型生成横幅的图像，结果表明该方法可以为用户创建高质量的个性化横幅。

文本到图像与大型语言模型的链式生成：一种新方法用于生成个性化电子商务横幅

Chaining text-to-image and large language model: A novel approach for  generating personalized e-commerce banners

Enhancing a still image with motion offers more engaged visual experience.
Traditional image animation techniques mainly focus on animating natural scenes
with random dynamics, such as clouds and fluid, and thus limits their
applicability to generic visual contents. To overcome this limitation, we
explore the synthesis of dynamic content for open-domain images, converting
them into animated videos. The key idea is to utilize the motion prior of
text-to-video diffusion models by incorporating the image into the generative
process as guidance. Given an image, we first project it into a text-aligned
rich image embedding space using a learnable image encoding network, which
facilitates the video model to digest the image content compatibly. However,
some visual details still struggle to be preserved in the resulting videos. To
supplement more precise image information, we further feed the full image to
the diffusion model by concatenating it with the initial noises. Experimental
results reveal that our proposed method produces visually convincing animated
videos, exhibiting both natural motions and high fidelity to the input image.
Comparative evaluation demonstrates the notable superiority of our approach
over existing competitors. The source code will be released upon publication.

通过将静止图像与动态相结合，提供更具吸引力的视觉体验。

DynamiCrafter: 用视频扩散先验动画化开放域图像

DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors

Homography estimation is an important step in many computer vision problems.
Recently, deep neural network methods have shown to be favorable for this
problem when compared to traditional methods. However, these new methods do not
consider dynamic content in input images. They train neural networks with only
image pairs that can be perfectly aligned using homographies. This paper
investigates and discusses how to design and train a deep neural network that
handles dynamic scenes. We first collect a large video dataset with dynamic
content. We then develop a multi-scale neural network and show that when
properly trained using our new dataset, this neural network can already handle
dynamic scenes to some extent. To estimate a homography of a dynamic scene in a
more principled way, we need to identify the dynamic content. Since dynamic
content detection and homography estimation are two tightly coupled tasks, we
follow the multi-task learning principles and augment our multi-scale network
such that it jointly estimates the dynamics masks and homographies. Our
experiments show that our method can robustly estimate homography for
challenging scenarios with dynamic scenes, blur artifacts, or lack of textures.

本篇论文研究并讨论如何设计和训练一种深度神经网络，以处理动态场景的单应性估计问题，通过多尺度网络进行设计和训练，在估计动态场景的单应性时，同时完成了动态内容检测的任务，使得该方法在具有挑战性的场景中具有较高的算法鲁棒性。