Social media plays a significant role in boosting the fashion industry, where
a massive amount of fashion-related posts are generated every day. In order to
obtain the rich fashion information from the posts, we study the task of social
media fashion knowledge extraction. Fashion knowledge, which typically consists
of the occasion, person attributes, and fashion item information, can be
effectively represented as a set of tuples. Most previous studies on fashion
knowledge extraction are based on the fashion product images without
considering the rich text information in social media posts. Existing work on
fashion knowledge extraction in social media is classification-based and
requires to manually determine a set of fashion knowledge categories in
advance. In our work, we propose to cast the task as a captioning problem to
capture the interplay of the multimodal post information. Specifically, we
transform the fashion knowledge tuples into a natural language caption with a
sentence transformation method. Our framework then aims to generate the
sentence-based fashion knowledge directly from the social media post. Inspired
by the big success of pre-trained models, we build our model based on a
multimodal pre-trained generative model and design several auxiliary tasks for
enhancing the knowledge extraction. Since there is no existing dataset which
can be directly borrowed to our task, we introduce a dataset consisting of
social media posts with manual fashion knowledge annotation. Extensive
experiments are conducted to demonstrate the effectiveness of our model.

社交媒体在时尚行业中起到了重要作用，本论文研究了社交媒体时尚知识抽取任务，将其作为一个描述问题来处理，旨在从社交媒体帖子中直接生成基于句子的时尚知识。