In this paper we address the task of summarizing television shows, which
touches key areas in AI research: complex reasoning, multiple modalities, and
long narratives. We present a modular approach where separate components
perform specialized sub-tasks which we argue affords greater flexibility
compared to end-to-end methods. Our modules involve detecting scene boundaries,
reordering scenes so as to minimize the number of cuts between different
events, converting visual information to text, summarizing the dialogue in each
scene, and fusing the scene summaries into a final summary for the entire
episode. We also present a new metric, PREFS (\textbf{P}recision and
\textbf{R}ecall \textbf{E}valuation of Summary \textbf{F}act\textbf{s}), to
measure both precision and recall of generated summaries, which we decompose
into atomic facts. Tested on the recently released SummScreen3D dataset
Papalampidi and Lapata (2023), our method produces higher quality summaries
than comparison models, as measured with ROUGE and our new fact-based metric.

这篇论文主要研究了如何用模块化方法总结电视节目，涉及人工智能研究中的复杂推理、多模态和长篇叙述等关键领域。作者提出了一种模块化方法，通过不同组件执行特定子任务，可以比端到端方法更灵活地实现。论文还引入了一个新的指标 —— 基于 PREFS（Summary Facts 的精确度与召回率评估）来衡量生成摘要的精确度与召回率，并将其分解为原子事实。在最近发布的 SummScreen3D 数据集上测试，该方法比其他模型生成的摘要具有更高的质量，这是通过 ROUGE 评估和新的基于事实的指标得出的。

电视节目的多模态摘要的模块化方法

A Modular Approach for Multimodal Summarization of TV Shows

Point cloud segmentation is fundamental in understanding 3D environments.
However, current 3D point cloud segmentation methods usually perform poorly on
scene boundaries, which degenerates the overall segmentation performance. In
this paper, we focus on the segmentation of scene boundaries. Accordingly, we
first explore metrics to evaluate the segmentation performance on scene
boundaries. To address the unsatisfactory performance on boundaries, we then
propose a novel contrastive boundary learning (CBL) framework for point cloud
segmentation. Specifically, the proposed CBL enhances feature discrimination
between points across boundaries by contrasting their representations with the
assistance of scene contexts at multiple scales. By applying CBL on three
different baseline methods, we experimentally show that CBL consistently
improves different baselines and assists them to achieve compelling performance
on boundaries, as well as the overall performance, eg in mIoU. The experimental
results demonstrate the effectiveness of our method and the importance of
boundaries for 3D point cloud segmentation. Code and model will be made
publicly available at this https URL

本文介绍了一种基于对比边界学习（CBL）的方法，在多个尺度上利用场景上下文来增强点之间边界对比的特征鉴别力，提高了 3D 点云分割的性能，尤其是在场景边界上的表现

点云分割的对比界限学习

Contrastive Boundary Learning for Point Cloud Segmentation

Scenes play a crucial role in breaking the storyline of movies and TV
episodes into semantically cohesive parts. However, given their complex
temporal structure, finding scene boundaries can be a challenging task
requiring large amounts of labeled training data. To address this challenge, we
present a self-supervised shot contrastive learning approach (ShotCoL) to learn
a shot representation that maximizes the similarity between nearby shots
compared to randomly selected shots. We show how to apply our learned shot
representation for the task of scene boundary detection to offer
state-of-the-art performance on the MovieNet dataset while requiring only ~25%
of the training labels, using 9x fewer model parameters and offering 7x faster
runtime. To assess the effectiveness of ShotCoL on novel applications of scene
boundary detection, we take on the problem of finding timestamps in movies and
TV episodes where video-ads can be inserted while offering a minimally
disruptive viewing experience. To this end, we collected a new dataset called
AdCuepoints with 3,975 movies and TV episodes, 2.2 million shots and 19,119
minimally disruptive ad cue-point labels. We present a thorough empirical
analysis on this dataset demonstrating the effectiveness of ShotCoL for ad
cue-points detection.

本文提出一个自监督的 ShotCoL 方法，其利用类比学习学习镜头表示，用于发现场景边界和广告插入时间。(This paper proposes a self-supervised ShotCoL method that utilizes contrastive learning to obtain shot representation for detecting scene boundaries and ad insertion timestamps.)