Fashion illustration is a crucial medium for designers to convey their
creative vision and transform design concepts into tangible representations
that showcase the interplay between clothing and the human body. In the context
of fashion design, computer vision techniques have the potential to enhance and
streamline the design process. Departing from prior research primarily focused
on virtual try-on, this paper tackles the task of multimodal-conditioned
fashion image editing. Our approach aims to generate human-centric fashion
images guided by multimodal prompts, including text, human body poses, garment
sketches, and fabric textures. To address this problem, we propose extending
latent diffusion models to incorporate these multiple modalities and modifying
the structure of the denoising network, taking multimodal prompts as input. To
condition the proposed architecture on fabric textures, we employ textual
inversion techniques and let diverse cross-attention layers of the denoising
network attend to textual and texture information, thus incorporating different
granularity conditioning details. Given the lack of datasets for the task, we
extend two existing fashion datasets, Dress Code and VITON-HD, with multimodal
annotations. Experimental evaluations demonstrate the effectiveness of our
proposed approach in terms of realism and coherence concerning the provided
multimodal inputs.

该论文探讨了多模态条件下的时尚图像编辑任务，提出了一种基于多模态指导的生成方法，利用时尚设计的文本、人体姿势、服装草图和面料纹理等多模态提示生成以人为中心的时尚图像。实验证明了提出方法在现实感和一致性方面的有效性。

时尚图像编辑的多模态条件化潜扩散模型

Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing

This paper explores active sensing strategies that employ vision-based
tactile sensors for robotic perception and classification of fabric textures.
We formalize the active sampling problem in the context of tactile fabric
recognition and provide an implementation of information-theoretic exploration
strategies based on minimizing predictive entropy and variance of probabilistic
models. Through ablation studies and human experiments, we investigate which
components are crucial for quick and reliable texture recognition. Along with
the active sampling strategies, we evaluate neural network architectures,
representations of uncertainty, influence of data augmentation, and dataset
variability. By evaluating our method on a previously published Active Clothing
Perception Dataset and on a real robotic system, we establish that the choice
of the active exploration strategy has only a minor influence on the
recognition accuracy, whereas data augmentation and dropout rate play a
significantly larger role. In a comparison study, while humans achieve 66.9%
recognition accuracy, our best approach reaches 90.0% in under 5 touches,
highlighting that vision-based tactile sensors are highly effective for fabric
texture recognition.

本研究旨在探索利用基于视觉的触觉传感器进行机器人感知和织物纹理分类的主动感知策略。通过信息理论探索策略的实现，以最小化概率模型的预测熵和方差，我们对触觉织物识别中的主动抽样问题进行了形式化。通过消融研究和人类实验，我们调查了对于快速可靠的纹理识别来说哪些组件至关重要。除了主动抽样策略外，我们还评估了神经网络架构、不确定性的表示、数据增强以及数据集的可变性。通过在先前发表的主动服装感知数据集和实际机器人系统上评估我们的方法，我们得出结论：主动探索策略的选择对识别准确性仅有较小影响，而数据增强和丢弃率则发挥更大的作用。通过对比研究，尽管人类识别准确度为 66.9%，但我们的最佳方法在不到 5 次触摸的情况下达到了 90.0%，突出表明基于视觉的触觉传感器对于纺织品纹理识别非常有效。