Due to the significant advances in large-scale text-to-image generation by
diffusion model (DM), controllable human image generation has been attracting
much attention recently. Existing works, such as Controlnet [36], T2I-adapter
[20] and HumanSD [10] have demonstrated good abilities in generating human
images based on pose conditions, they still fail to meet the requirements of
real e-commerce scenarios. These include (1) the interaction between the shown
product and human should be considered, (2) human parts like face/hand/arm/foot
and the interaction between human model and product should be hyper-realistic,
and (3) the identity of the product shown in advertising should be exactly
consistent with the product itself. To this end, in this paper, we first define
a new human image generation task for e-commerce marketing, i.e.,
Object-ID-retentive Human-object Interaction image Generation (OHG), and then
propose a VirtualModel framework to generate human images for product shown,
which supports displays of any categories of products and any types of
human-object interaction. As shown in Figure 1, VirtualModel not only
outperforms other methods in terms of accurate pose control and image quality
but also allows for the display of user-specified product objects by
maintaining the product-ID consistency and enhancing the plausibility of
human-object interaction. Codes and data will be released.

本文提出了一种用于电子商务营销的虚拟模型框架，旨在生成与产品完全一致的人物图像，并增强人物与产品的合理性和真实性。

VirtualModel：通过扩散模型生成具有对象识别保持性的人 - 对象互动图片以用于电子商务营销

VirtualModel: Generating Object-ID-retentive Human-object Interaction  Image by Diffusion Model for E-commerce Marketing

With the development of the large-scale diffusion model, Artificial
Intelligence Generated Content (AIGC) techniques are popular recently. However,
how to truly make it serve our daily lives remains an open question. To this
end, in this paper, we focus on employing AIGC techniques in one filed of
E-commerce marketing, i.e., generating hyper-realistic advertising images for
displaying user-specified shoes by human. Specifically, we propose a
shoe-wearing system, called Shoe-Model, to generate plausible images of human
legs interacting with the given shoes. It consists of three modules: (1) shoe
wearable-area detection module (WD), (2) leg-pose synthesis module (LpS) and
the final (3) shoe-wearing image generation module (SW). Them three are
performed in ordered stages. Compared to baselines, our ShoeModel is shown to
generalize better to different type of shoes and has ability of keeping the
ID-consistency of the given shoes, as well as automatically producing
reasonable interactions with human. Extensive experiments show the
effectiveness of our proposed shoe-wearing system. Figure 1 shows the input and
output examples of our ShoeModel.

本文提出了一种名为 Shoe-Model 的穿鞋系统，利用人工智能生成内容技术在电子商务营销领域生成超逼真的广告图像，为用户指定的鞋子生成与人体互动的逼真图像。通过三个模块：鞋子可穿戴区域检测模块（WD）、腿部姿势合成模块（LpS）和最终的鞋穿图像生成模块（SW），在有序的阶段中完成。与基线相比，我们的 ShoeModel 对不同类型的鞋子具有更好的泛化能力，并具有保持给定鞋子 ID 一致性以及自动产生合理人体互动的能力，大量实验表明了我们所提出的穿鞋系统的有效性。