Given a clothing image and a person image, an image-based virtual try-on aims to generate a customized image that appears natural and accurately reflects the characteristics of the clothing image. In this work, we aim to expand the applicability of the pre-trained diffusion model so that it can be utilized independently for the virtual try-on task.The main challenge is to preserve the clothing details while effectively utilizing the robust generative capability of the pre-trained model. In order to tackle these issues, we propose StableVITON, learning the semantic correspondence between the clothing and the human body within the latent space of the pre-trained diffusion model in an end-to-end manner. Our proposed zero cross-attention blocks not only preserve the clothing details by learning the semantic correspondence but also generate high-fidelity images by utilizing the inherent knowledge of the pre-trained model in the warping process. Through our proposed novel attention total variation loss and applying augmentation, we achieve the sharp attention map, resulting in a more precise representation of clothing details. StableVITON outperforms the baselines in qualitative and quantitative evaluation, showing promising quality in arbitrary person images. Our code is available at https://github.com/rlawjdghek/StableVITON.

通过在预训练扩散模型的潜在空间内学习衣物与人体之间的语义对应关系，利用零交叉关注区块保留衣物细节并利用预训练模型的内在知识进行图像变形处理，通过提出的新型注意力全变差损失和应用数据增强，实现了尖锐的注意力图，从而更准确地表示衣物细节。StableVITON在定性和定量评估中胜过基准方法，在任意人物图像中展示了有希望的质量。

StableVITON：使用潜在扩散模型学习虚拟试穿的语义对应关系