Implicit neural representations (INR) have gained increasing attention in
representing 3D scenes and images, and have been recently applied to encode
videos (e.g., NeRV, E-NeRV). While achieving promising results, existing
INR-based methods are limited to encoding a handful of short videos (e.g.,
seven 5-second videos in the UVG dataset) with redundant visual content,
leading to a model design that fits individual video frames independently and
is not efficiently scalable to a large number of diverse videos. This paper
focuses on developing neural representations for a more practical setup --
encoding long and/or a large number of videos with diverse visual content. We
first show that instead of dividing videos into small subsets and encoding them
with separate models, encoding long and diverse videos jointly with a unified
model achieves better compression results. Based on this observation, we
propose D-NeRV, a novel neural representation framework designed to encode
diverse videos by (i) decoupling clip-specific visual content from motion
information, (ii) introducing temporal reasoning into the implicit neural
network, and (iii) employing the task-oriented flow as intermediate output to
reduce spatial redundancies. Our new model largely surpasses NeRV and
traditional video compression techniques on UCF101 and UVG datasets on the
video compression task. Moreover, when used as an efficient data-loader, D-NeRV
achieves 3%-10% higher accuracy than NeRV on action recognition tasks on the
UCF101 dataset under the same compression ratios.

该研究提出了一种新的神经表示框架 D-NeRV，旨在以更高效的方式对长时间或大量的不同内容的视频进行编码。使用这种方法，该方法的压缩结果大大超过了现有的 NeRV 和传统视频压缩技术，同时在相同的压缩率下，其在 UCF101 数据集上用作高效数据加载器时的准确度比 NeRV 提高了 3 至 10％。