This paper introduces MMTryon, a multi-modal multi-reference VIrtual Try-ON
(VITON) framework, which can generate high-quality compositional try-on results
by taking as inputs a text instruction and multiple garment images. Our MMTryon
mainly addresses two problems overlooked in prior literature: 1) Support of
multiple try-on items and dressing styleExisting methods are commonly designed
for single-item try-on tasks (e.g., upper/lower garments, dresses) and fall
short on customizing dressing styles (e.g., zipped/unzipped, tuck-in/tuck-out,
etc.) 2) Segmentation Dependency. They further heavily rely on
category-specific segmentation models to identify the replacement regions, with
segmentation errors directly leading to significant artifacts in the try-on
results. For the first issue, our MMTryon introduces a novel multi-modality and
multi-reference attention mechanism to combine the garment information from
reference images and dressing-style information from text instructions.
Besides, to remove the segmentation dependency, MMTryon uses a parsing-free
garment encoder and leverages a novel scalable data generation pipeline to
convert existing VITON datasets to a form that allows MMTryon to be trained
without requiring any explicit segmentation. Extensive experiments on
high-resolution benchmarks and in-the-wild test sets demonstrate MMTryon's
superiority over existing SOTA methods both qualitatively and quantitatively.
Besides, MMTryon's impressive performance on multi-items and style-controllable
virtual try-on scenarios and its ability to try on any outfit in a large
variety of scenarios from any source image, opens up a new avenue for future
investigation in the fashion community.

MMTryon 是一个多模态多参考虚拟试衣 (VITON) 框架，通过输入文本指令和多个服装图像，可以生成高质量的组合试穿结果。它通过引入新颖的多模态和多参考注意机制来解决现有研究中忽略的多个试穿物品和定制穿着风格的问题，并使用无解析的服装编码器和新颖的可扩展数据生成流水线来消除对分割的依赖。实验证明了 MMTryon 在定性和定量方面相对于现有的技术的卓越表现，为时尚界探索多物品和可控风格虚拟试衣场景打开新的研究方向。