Social media plays a significant role in boosting the fashion industry, where
a massive amount of fashion-related posts are generated every day. In order to
obtain the rich fashion information from the posts, we study the task of social
media fashion knowledge extraction. Fashion knowledge, which typically consists
of the occasion, person attributes, and fashion item information, can be
effectively represented as a set of tuples. Most previous studies on fashion
knowledge extraction are based on the fashion product images without
considering the rich text information in social media posts. Existing work on
fashion knowledge extraction in social media is classification-based and
requires to manually determine a set of fashion knowledge categories in
advance. In our work, we propose to cast the task as a captioning problem to
capture the interplay of the multimodal post information. Specifically, we
transform the fashion knowledge tuples into a natural language caption with a
sentence transformation method. Our framework then aims to generate the
sentence-based fashion knowledge directly from the social media post. Inspired
by the big success of pre-trained models, we build our model based on a
multimodal pre-trained generative model and design several auxiliary tasks for
enhancing the knowledge extraction. Since there is no existing dataset which
can be directly borrowed to our task, we introduce a dataset consisting of
social media posts with manual fashion knowledge annotation. Extensive
experiments are conducted to demonstrate the effectiveness of our model.

社交媒体在时尚行业中起到了重要作用，本论文研究了社交媒体时尚知识抽取任务，将其作为一个描述问题来处理，旨在从社交媒体帖子中直接生成基于句子的时尚知识。

社交媒体时尚知识提取

Social Media Fashion Knowledge Extraction as Captioning

Fashion knowledge plays a pivotal role in helping people in their dressing.
In this paper, we present a novel system to automatically harvest fashion
knowledge from social media. It unifies three tasks of occasion, person and
clothing discovery from multiple modalities of images, texts and metadata. A
contextualized fashion concept learning model is applied to leverage the rich
contextual information for improving the fashion concept learning performance.
At the same time, to counter the label noise within training data, we employ a
weak label modeling method to further boost the performance. We build a website
to demonstrate the quality of fashion knowledge extracted by our system.

本文提出了一种从社交媒体自动收集时尚知识的新系统，它通过图像、文本和元数据的多个模态统一了场合、人物和服装发现的三个任务，并应用上下文化的时尚概念学习模型来提高时尚概念学习性能，同时通过弱标签建模方法进一步提高性能，并构建了一个网站来展示我们的系统提取的时尚知识的质量。

社交媒体中的时尚知识自动提取

Automatic Fashion Knowledge Extraction from Social Media

Fashion knowledge helps people to dress properly and addresses not only
physiological needs of users, but also the demands of social activities and
conventions. It usually involves three mutually related aspects of: occasion,
person and clothing. However, there are few works focusing on extracting such
knowledge, which will greatly benefit many downstream applications, such as
fashion recommendation. In this paper, we propose a novel method to
automatically harvest fashion knowledge from social media. We unify three tasks
of occasion, person and clothing discovery from multiple modalities of images,
texts and metadata. For person detection and analysis, we use the off-the-shelf
tools due to their flexibility and satisfactory performance. For clothing
recognition and occasion prediction, we unify the two tasks by using a
contextualized fashion concept learning module, which captures the dependencies
and correlations among different fashion concepts. To alleviate the heavy
burden of human annotations, we introduce a weak label modeling module which
can effectively exploit machine-labeled data, a complementary of clean data. In
experiments, we contribute a benchmark dataset and conduct extensive
experiments from both quantitative and qualitative perspectives. The results
demonstrate the effectiveness of our model in fashion concept prediction, and
the usefulness of extracted knowledge with comprehensive analysis.

本篇论文提出了一种从社交媒体中自动提取时尚知识的方法，该方法结合了图像、文本和元数据多种模态中的场合、人物和服装三个方面进行探测，并采用上下文化时尚概念学习模块提供了对于人物检测和服装识别的解决方案，在机器标注数据有限的情况下，通过弱标签建模模块有效地利用了机器标注数据。实验结果证明了我们的模型在时尚概念预测方面的有效性以及提取知识与全面分析的有用性。