In this paper, we present frame reconstruction model: FrameRS. It consists
self-supervised video frame reconstructor and key frame selector. The frame
reconstructor, FrameMAE, is developed by adapting the principles of the Masked
Autoencoder for Images (MAE) for video context. The key frame selector, Frame
Selector, is built on CNN architecture. By taking the high-level semantic
information from the encoder of FrameMAE as its input, it can predicted the key
frames with low computation costs. Integrated with our bespoke Frame Selector,
FrameMAE can effectively compress a video clip by retaining approximately 30%
of its pivotal frames. Performance-wise, our model showcases computational
efficiency and competitive accuracy, marking a notable improvement over
traditional Key Frame Extract algorithms. The implementation is available on
Github

本研究提出了一个名为 FrameRS 的帧重建模型。它由自监督的视频帧重建器 FrameMAE 和关键帧选择器 Frame Selector 组成。FrameMAE 采用了 Masked Autoencoder for Images（MAE）的原理用于视频环境。Frame Selector 则基于 CNN 架构，并利用来自 FrameMAE 编码器的高层语义信息作为输入，可以以较低的计算成本预测关键帧。我们的模型集成了 Frame Selector 和 FrameMAE，可以有效地压缩视频片段，保留其关键帧的约 30%。在性能方面，我们的模型展示了计算效率和竞争性的准确性，显著改进了传统的关键帧提取算法。该模型的实现可在 Github 上找到。