In this work, we first propose a fully differentiable Many-to-Many (M2M)
splatting framework to interpolate frames efficiently. Given a frame pair, we
estimate multiple bidirectional flows to directly forward warp the pixels to
the desired time step before fusing overlapping pixels. In doing so, each
source pixel renders multiple target pixels and each target pixel can be
synthesized from a larger area of visual context, establishing a many-to-many
splatting scheme with robustness to undesirable artifacts. For each input frame
pair, M2M has a minuscule computational overhead when interpolating an
arbitrary number of in-between frames, hence achieving fast multi-frame
interpolation. However, directly warping and fusing pixels in the intensity
domain is sensitive to the quality of motion estimation and may suffer from
less effective representation capacity. To improve interpolation accuracy, we
further extend an M2M++ framework by introducing a flexible Spatial Selective
Refinement (SSR) component, which allows for trading computational efficiency
for interpolation quality and vice versa. Instead of refining the entire
interpolated frame, SSR only processes difficult regions selected under the
guidance of an estimated error map, thereby avoiding redundant computation.
Evaluation on multiple benchmark datasets shows that our method is able to
improve the efficiency while maintaining competitive video interpolation
quality, and it can be adjusted to use more or less compute as needed.

我们提出了一个全可微的 Many-to-Many（M2M）splatting 框架，通过估计多个双向流来直接向前扭曲像素到所需的时间步骤，实现了多对多的覆盖像素的效果，然后引入了一个灵活的空间选择性优化（SSR）模块来提高插值精度，从而提高了视频插值的效率和质量。