Virtual 3D try-on can provide an intuitive and realistic view for online
shopping and has a huge potential commercial value. However, existing 3D
virtual try-on methods mainly rely on annotated 3D human shapes and garment
templates, which hinders their applications in practical scenarios. 2D virtual
try-on approaches provide a faster alternative to manipulate clothed humans,
but lack the rich and realistic 3D representation. In this paper, we propose a
novel Monocular-to-3D Virtual Try-On Network (M3D-VTON) that builds on the
merits of both 2D and 3D approaches. By integrating 2D information efficiently
and learning a mapping that lifts the 2D representation to 3D, we make the
first attempt to reconstruct a 3D try-on mesh only taking the target clothing
and a person image as inputs. The proposed M3D-VTON includes three modules: 1)
The Monocular Prediction Module (MPM) that estimates an initial full-body depth
map and accomplishes 2D clothes-person alignment through a novel two-stage
warping procedure; 2) The Depth Refinement Module (DRM) that refines the
initial body depth to produce more detailed pleat and face characteristics; 3)
The Texture Fusion Module (TFM) that fuses the warped clothing with the
non-target body part to refine the results. We also construct a high-quality
synthesized Monocular-to-3D virtual try-on dataset, in which each person image
is associated with a front and a back depth map. Extensive experiments
demonstrate that the proposed M3D-VTON can manipulate and reconstruct the 3D
human body wearing the given clothing with compelling details and is more
efficient than other 3D approaches.

该研究提出一种新颖的基于 2D 和 3D 方法的单目 - 3D 虚拟试穿网络，通过有效地整合 2D 信息和学习将 2D 表示提升到 3D 的映射以构建一个 3D 试穿模型。所提出的模型包含三个模块：单目预测模块、深度细化模块和纹理融合模块。该模型在构建高质量的试衣数据集的基础上，通过大量实验，证明了其比其他 3D 方法更高效和能够更真实地描述试穿效果。