Previous research has studied the task of segmenting cinematic videos into scenes and into narrative acts. However, these studies have overlooked the essential task of multimodal alignment and fusion for effectively and efficiently processing long-form videos (>60min). In this paper, we introduce Multimodal alignmEnt aGgregation and distillAtion (MEGA) for cinematic long-video segmentation. MEGA tackles the challenge by leveraging multiple media modalities. The method coarsely aligns inputs of variable lengths and different modalities with alignment positional encoding. To maintain temporal synchronization while reducing computation, we further introduce an enhanced bottleneck fusion layer which uses temporal alignment. Additionally, MEGA employs a novel contrastive loss to synchronize and transfer labels across modalities, enabling act segmentation from labeled synopsis sentences on video shots. Our experimental results show that MEGA outperforms state-of-the-art methods on MovieNet dataset for scene segmentation (with an Average Precision improvement of +1.19%) and on TRIPOD dataset for act segmentation (with a Total Agreement improvement of +5.51%)

本研究介绍了一种用于电影长视频分割的多模态对齐聚合与提取的方法（MEGA），该方法通过利用多个媒体模态来粗略对齐不同长度和不同模态的输入，并采用增强的瓶颈融合层和对比损失来保持时间同步并实现跨模态的标签转移与段落划分。实验结果表明，MEGA在场景分割和片段分割方面优于现有的方法，在MovieNet数据集上平均精确度提高了1.19％，在TRIPOD数据集上总一致性提高了5.51％。

MEGA: 电影视频分割的多模态对齐、聚合与蒸馏