Generating an informative and attractive title for the product is a crucial
task for e-commerce. Most existing works follow the standard multimodal natural
language generation approaches, e.g., image captioning, and employ the large
scale of human-labelled datasets to train desirable models. However, for novel
products, especially in a different domain, there are few existing labelled
data. In this paper, we propose a prompt-based approach, i.e., the Multimodal
Prompt Learning framework, to accurately and efficiently generate titles for
novel products with limited labels. We observe that the core challenges of
novel product title generation are the understanding of novel product
characteristics and the generation of titles in a novel writing style. To this
end, we build a set of multimodal prompts from different modalities to preserve
the corresponding characteristics and writing styles of novel products. As a
result, with extremely limited labels for training, the proposed method can
retrieve the multimodal prompts to generate desirable titles for novel
products. The experiments and analyses are conducted on five novel product
categories under both the in-domain and out-of-domain experimental settings.
The results show that, with only 1% of downstream labelled data for training,
our proposed approach achieves the best few-shot results and even achieves
competitive results with fully-supervised methods trained on 100% of training
data; With the full labelled data for training, our method achieves
state-of-the-art results.

本文提出了一种基于多模态提示学习框架的方法，通过仅使用少量标注数据快速高效地为新颖产品生成标题，并在不同场景中进行实验证明，在仅使用 downstream 标签数据 1% 的情况下，该方法实现了最佳 few-shot 学习效果，甚至与完全监督的方法相比也具有竞争力。

极少标签下的多模态提示学习用于产品标题生成

Multimodal Prompt Learning for Product Title Generation with Extremely  Limited Labels

Nowadays, more and more customers browse and purchase products in favor of
using mobile E-Commerce Apps such as Taobao and Amazon. Since merchants are
usually inclined to describe redundant and over-informative product titles to
attract attentions from customers, it is important to concisely display short
product titles on limited screen of mobile phones. To address this discrepancy,
previous studies mainly consider textual information of long product titles and
lacks of human-like view during training and evaluation process. In this paper,
we propose a Multi-Modal Generative Adversarial Network (MM-GAN) for short
product title generation in E-Commerce, which innovatively incorporates image
information and attribute tags from product, as well as textual information
from original long titles. MM-GAN poses short title generation as a
reinforcement learning process, where the generated titles are evaluated by the
discriminator in a human-like view. Extensive experiments on a large-scale
E-Commerce dataset demonstrate that our algorithm outperforms other
state-of-the-art methods. Moreover, we deploy our model into a real-world
online E-Commerce environment and effectively boost the performance of click
through rate and click conversion rate by 1.66% and 1.87%, respectively.

本文提出了一种用于移动电商下的产品标题生成算法 MM-GAN，该算法从原始长标题中提取文本信息、产品的图像信息和属性标签等多模态信息，并将短标题生成视为一个强化学习过程。在进行模型训练和评估时，以类人的视角评估生成的标题，该算法在大规模电商数据集上的实验证明，与其他方法相比，本算法表现更优，应用于真实世界的电商平台后，点击率和点击转化率均有明显提高。