Multimedia summarization with multimodal output can play an essential role in
real-world applications, i.e., automatically generating cover images and titles
for news articles or providing introductions to online videos. In this work, we
propose a multimodal hierarchical multimedia summarization (MHMS) framework by
interacting visual and language domains to generate both video and textual
summaries. Our MHMS method contains video and textual segmentation and
summarization module, respectively. It formulates a cross-domain alignment
objective with optimal transport distance which leverages cross-domain
interaction to generate the representative keyframe and textual summary. We
evaluated MHMS on three recent multimodal datasets and demonstrated the
effectiveness of our method in producing high-quality multimodal summaries.

本文提出一个交互视觉和语言领域的多模式多媒体总结（MHMS）框架，旨在通过多模式输出为新闻文章自动生成封面图片和标题，或为在线视频提供介绍。本文的 MHMS 方法包含视频和文本细分和摘要模块，分别生成代表性关键帧和文本摘要，并利用最优输运距离制定跨域对齐目标，从而增强了跨域交互并产生高质量的多模式摘要。本文在三个最近发布的多模式数据集上评估了 MHMS 方法的效果。