We address the problem of highlight detection from a 360 degree video by summarizing it both spatially and temporally. Given a long 360 degree video, we spatially select pleasantly-looking normal field-of-view (NFOV) segments from unlimited field of views (FOV) of the 360 degree video, and temporally summarize it into a concise and informative highlight as a selected subset of subshots. We propose a novel deep ranking model named as Composition View Score (CVS) model, which produces a spherical score map of composition per video segment, and determines which view is suitable for highlight via a sliding window kernel at inference. To evaluate the proposed framework, we perform experiments on the Pano2Vid benchmark dataset and our newly collected 360 degree video highlight dataset from YouTube and Vimeo. Through evaluation using both quantitative summarization metrics and user studies via Amazon Mechanical Turk, we demonstrate that our approach outperforms several state-of-the-art highlight detection methods. We also show that our model is 16 times faster at inference than AutoCam, which is one of the first summarization algorithms of 360 degree videos

本文提出了一种基于空间和时间摘要的方法来从长视频中检测高光部分。通过从无限视角中选择宜人的普通视野(NFOV)段落并将其在时间上进行总结，选择子镜头的子集来形成简明的和信息丰富的高光。研究人员提出了一种名为组合视图分数(CVS)模型的新型深度排名模型，该模型为每个视频段的组合生成一个球面分数图，并在推理时通过滑动窗口内核确定哪个视图适用于高光。通过在Pano2Vid基准数据集和从YouTube和Vimeo收集的我们新收集的360度视频亮点数据集上评估所提出的框架，我们进行了实验。通过使用定量总结度量和通过Amazon Mechanical Turk的用户研究进行评估，我们证明了我们的方法优于几种现有的高光检测方法。同时，我们还展示了我们的模型在推理速度上比AutoCam(360视频摘要的第一个算法之一)快16倍。

一种用于360度视频时空精彩片段检测的深度排名模型