Video summarization aims to select representative frames to retain high-level
information, which is usually solved by predicting the segment-wise importance
score via a softmax function. However, softmax function suffers in retaining
high-rank representations for complex visual or sequential information, which
is known as the Softmax Bottleneck problem. In this paper, we propose a novel
framework named Dual Mixture Attention (DMASum) model with Meta Learning for
video summarization that tackles the softmax bottleneck problem, where the
Mixture of Attention layer (MoA) effectively increases the model capacity by
employing twice self-query attention that can capture the second-order changes
in addition to the initial query-key attention, and a novel Single Frame Meta
Learning rule is then introduced to achieve more generalization to small
datasets with limited training sources. Furthermore, the DMASum significantly
exploits both visual and sequential attention that connects local key-frame and
global attention in an accumulative way. We adopt the new evaluation protocol
on two public datasets, SumMe, and TVSum. Both qualitative and quantitative
experiments manifest significant improvements over the state-of-the-art
methods.

本文提出了一种新的框架，即 Dual Mixture Attention (DMASum) 模型，其中 Mixture of Attention layer (MoA) 通过两次自己 - 查询注意实现，可以捕获二阶变化以及初始查询 - 键注意，进而有效增加视频摘要中的模型能力，在引入新的单帧元学习规则后更广泛地实现小数据集的泛化，并且 DMASum 通过累积方式连接本地关键帧和全局关注，显著提高了 SumMe 和 TVSum 这两个公共数据集的定性和定量实验结果，解决了 softmax 瓶颈问题。