Video virtual try-on aims to transfer a clothing item onto the video of a
target person. Directly applying the technique of image-based try-on to the
video domain in a frame-wise manner will cause temporal-inconsistent outcomes
while previous video-based try-on solutions can only generate low visual
quality and blurring results. In this work, we present ViViD, a novel framework
employing powerful diffusion models to tackle the task of video virtual try-on.
Specifically, we design the Garment Encoder to extract fine-grained clothing
semantic features, guiding the model to capture garment details and inject them
into the target video through the proposed attention feature fusion mechanism.
To ensure spatial-temporal consistency, we introduce a lightweight Pose Encoder
to encode pose signals, enabling the model to learn the interactions between
clothing and human posture and insert hierarchical Temporal Modules into the
text-to-image stable diffusion model for more coherent and lifelike video
synthesis. Furthermore, we collect a new dataset, which is the largest, with
the most diverse types of garments and the highest resolution for the task of
video virtual try-on to date. Extensive experiments demonstrate that our
approach is able to yield satisfactory video try-on results. The dataset,
codes, and weights will be publicly available. Project page:
this https URL

视频虚拟试穿通过强大的扩散模型处理视频虚拟试穿任务并实现空间时间一致性。