Despite their impressive generative performance, latent diffusion model-based
virtual try-on (VTON) methods lack faithfulness to crucial details of the
clothes, such as style, pattern, and text. To alleviate these issues caused by
the diffusion stochastic nature and latent supervision, we propose a novel
Faithful Latent Diffusion Model for VTON, termed FLDM-VTON. FLDM-VTON improves
the conventional latent diffusion process in three major aspects. First, we
propose incorporating warped clothes as both the starting point and local
condition, supplying the model with faithful clothes priors. Second, we
introduce a novel clothes flattening network to constrain generated try-on
images, providing clothes-consistent faithful supervision. Third, we devise a
clothes-posterior sampling for faithful inference, further enhancing the model
performance over conventional clothes-agnostic Gaussian sampling. Extensive
experimental results on the benchmark VITON-HD and Dress Code datasets
demonstrate that our FLDM-VTON outperforms state-of-the-art baselines and is
able to generate photo-realistic try-on images with faithful clothing details.

虽然虚拟试衣（VTON）基于潜在扩散模型的生成性能令人印象深刻，但缺乏对衣物关键细节（如风格、图案和文字）的忠实性。为了缓解由扩散随机性和潜在监督引起的这些问题，我们提出了一种新颖的面向 VTON 的忠实潜在扩散模型，称为 FLDM-VTON。FLDM-VTON 在三个主要方面改进了传统的潜在扩散过程。首先，我们提议将变形的衣物作为起点和局部条件，为模型提供忠实的衣物先验。其次，我们引入了一种新颖的衣物扁平化网络，以约束生成的试衣图像，提供一致的衣物忠实监督。第三，我们设计了一种具有忠实推理的衣物后验采样，进一步提高了模型在传统的衣物不可知高斯采样上的性能。对基准 VITON-HD 和 Dress Code 数据集的大量实验证明，我们的 FLDM-VTON 优于最先进的基线方法，并能够生成具有忠实衣物细节的照片级别的试衣图像。

FLDM-VTON：用于虚拟试衣的忠实潜在扩散模型

FLDM-VTON: Faithful Latent Diffusion Model for Virtual Try-on

Given a clothing image and a person image, an image-based virtual try-on aims
to generate a customized image that appears natural and accurately reflects the
characteristics of the clothing image. In this work, we aim to expand the
applicability of the pre-trained diffusion model so that it can be utilized
independently for the virtual try-on task.The main challenge is to preserve the
clothing details while effectively utilizing the robust generative capability
of the pre-trained model. In order to tackle these issues, we propose
StableVITON, learning the semantic correspondence between the clothing and the
human body within the latent space of the pre-trained diffusion model in an
end-to-end manner. Our proposed zero cross-attention blocks not only preserve
the clothing details by learning the semantic correspondence but also generate
high-fidelity images by utilizing the inherent knowledge of the pre-trained
model in the warping process. Through our proposed novel attention total
variation loss and applying augmentation, we achieve the sharp attention map,
resulting in a more precise representation of clothing details. StableVITON
outperforms the baselines in qualitative and quantitative evaluation, showing
promising quality in arbitrary person images. Our code is available at
this https URL

通过在预训练扩散模型的潜在空间内学习衣物与人体之间的语义对应关系，利用零交叉关注区块保留衣物细节并利用预训练模型的内在知识进行图像变形处理，通过提出的新型注意力全变差损失和应用数据增强，实现了尖锐的注意力图，从而更准确地表示衣物细节。StableVITON 在定性和定量评估中胜过基准方法，在任意人物图像中展示了有希望的质量。

StableVITON：使用潜在扩散模型学习虚拟试穿的语义对应关系

StableVITON: Learning Semantic Correspondence with Latent Diffusion  Model for Virtual Try-On

Image-based virtual try-on systems for fitting new in-shop clothes into a
person image have attracted increasing research attention, yet is still
challenging. A desirable pipeline should not only transform the target clothes
into the most fitting shape seamlessly but also preserve well the clothes
identity in the generated image, that is, the key characteristics (e.g.
texture, logo, embroidery) that depict the original clothes. However, previous
image-conditioned generation works fail to meet these critical requirements
towards the plausible virtual try-on performance since they fail to handle
large spatial misalignment between the input image and target clothes. Prior
work explicitly tackled spatial deformation using shape context matching, but
failed to preserve clothing details due to its coarse-to-fine strategy. In this
work, we propose a new fully-learnable Characteristic-Preserving Virtual Try-On
Network(CP-VTON) for addressing all real-world challenges in this task. First,
CP-VTON learns a thin-plate spline transformation for transforming the in-shop
clothes into fitting the body shape of the target person via a new Geometric
Matching Module (GMM) rather than computing correspondences of interest points
as prior works did. Second, to alleviate boundary artifacts of warped clothes
and make the results more realistic, we employ a Try-On Module that learns a
composition mask to integrate the warped clothes and the rendered image to
ensure smoothness. Extensive experiments on a fashion dataset demonstrate our
CP-VTON achieves the state-of-the-art virtual try-on performance both
qualitatively and quantitatively.

本文提出了一种基于图像的虚拟试衣系统，使用全新的几何匹配模块进行形变和修补，在保留衣物特征的同时，同时处理衣物和图像之间的边界问题和计算拟合度，实现了真实感和模拟性能的完美融合。