Slow motion videos are becoming increasingly popular, but capturing
high-resolution videos at extremely high frame rates requires professional
high-speed cameras. To mitigate this problem, current techniques increase the
frame rate of standard videos through frame interpolation by assuming linear
object motion which is not valid in challenging cases. In this paper, we
address this problem using two video streams as input; an auxiliary video with
high frame rate and low spatial resolution, providing temporal information, in
addition to the standard main video with low frame rate and high spatial
resolution. We propose a two-stage deep learning system consisting of alignment
and appearance estimation that reconstructs high resolution slow motion video
from the hybrid video input. For alignment, we propose to compute flows between
the missing frame and two existing frames of the main video by utilizing the
content of the auxiliary video frames. For appearance estimation, we propose to
combine the warped and auxiliary frames using a context and occlusion aware
network. We train our model on synthetically generated hybrid videos and show
high-quality results on a variety of test scenes. To demonstrate practicality,
we show the performance of our system on two real dual camera setups with small
baseline.

使用双流视频输入提出了一个由两个阶段的深度学习系统组成的框架，包含对齐和外观估计，可以从混合视频输入中重构高分辨率的慢动作视频。

混合成像系统实现的深度缓慢视频重建

Deep Slow Motion Video Reconstruction with Hybrid Imaging System

We present a novel end-to-end single-shot method that segments countable
object instances (things) as well as background regions (stuff) into a
non-overlapping panoptic segmentation at almost video frame rate. Current
state-of-the-art methods are far from reaching video frame rate and mostly rely
on merging instance segmentation with semantic background segmentation, making
them impractical to use in many applications such as robotics. Our approach
relaxes this requirement by using an object detector but is still able to
resolve inter- and intra-class overlaps to achieve a non-overlapping
segmentation. On top of a shared encoder-decoder backbone, we utilize multiple
branches for semantic segmentation, object detection, and instance center
prediction. Finally, our panoptic head combines all outputs into a panoptic
segmentation and can even handle conflicting predictions between branches as
well as certain false predictions. Our network achieves 32.6% PQ on MS-COCO at
23.5 FPS, opening up panoptic segmentation to a broader field of applications.

本文提出了一种全新的端到端单次方法，可以将可计数的物体实例 (事物) 以及背景区域 (物料) 分段成为一个几乎达到视频帧率的非重叠全景分割。