With the widespread popularity of user-generated short videos, it becomes
increasingly challenging for content creators to promote their content to
potential viewers. Automatically generating appealing titles and covers for
short videos can help grab viewers' attention. Existing studies on video
captioning mostly focus on generating factual descriptions of actions, which do
not conform to video titles intended for catching viewer attention.
Furthermore, research for cover selection based on multimodal information is
sparse. These problems motivate the need for tailored methods to specifically
support the joint task of short video title generation and cover selection
(TG-CS) as well as the demand for creating corresponding datasets to support
the studies. In this paper, we first collect and present a real-world dataset
named Short Video Title Generation (SVTG) that contains videos with appealing
titles and covers. We then propose a Title generation and Cover selection with
attention Refinement (TCR) method for TG-CS. The refinement procedure
progressively selects high-quality samples and highly relevant frames and text
tokens within each sample to refine model training. Extensive experiments show
that our TCR method is superior to various existing video captioning methods in
generating titles and is able to select better covers for noisy real-world
short videos.

本文介绍了一种名为 Short Video Title Generation (SVTG) 的真实数据集，以及一种应用多模态信息的 Title generation and Cover selection with attention Refinement (TCR) 方法，通过对高质量样本和高相关性帧和文本令牌进行精细化训练，该方法在生成标题和选择封面方面优于现有的各种视频字幕方法，并且可以针对真实短视频选择更好的封面。

TCR: 基于注意力细化的短视频标题生成及封面选择

TCR: Short Video Title Generation and Cover Selection with Attention  Refinement

A great video title describes the most salient event compactly and captures
the viewer's attention. In contrast, video captioning tends to generate
sentences that describe the video as a whole. Although generating a video title
automatically is a very useful task, it is much less addressed than video
captioning. We address video title generation for the first time by proposing
two methods that extend state-of-the-art video captioners to this new task.
First, we make video captioners highlight sensitive by priming them with a
highlight detector. Our framework allows for jointly training a model for title
generation and video highlight localization. Second, we induce high sentence
diversity in video captioners, so that the generated titles are also diverse
and catchy. This means that a large number of sentences might be required to
learn the sentence structure of titles. Hence, we propose a novel sentence
augmentation method to train a captioner with additional sentence-only examples
that come without corresponding videos. We collected a large-scale Video Titles
in the Wild (VTW) dataset of 18100 automatically crawled user-generated videos
and titles. On VTW, our methods consistently improve title prediction accuracy,
and achieve the best performance in both automatic and human evaluation.
Finally, our sentence augmentation method also outperforms the baselines on the
M-VAD dataset.

本研究提出了两种方法扩展现有视频的字幕生成模型以生成视频标题，即通过高亮检测器使视频字幕生成模型具有敏感性并同时进行标题生成和视频亮点定位训练，以及通过增强句子多样性，用大量无对应视频的句子训练模型，并在 18100 个用户生成的视频和标题（VTW）数据集和 M-VAD 数据集上验证了本研究的方法，均获得了最佳效果。