We describe an approach used in the Generic Boundary Event Captioning challenge at the Long-Form Video Understanding Workshop held at CVPR 2022. We designed a Rich Encoder-decoder framework for Video Event CAptioner (REVECA) that utilizes spatial and temporal information from the video to generate a caption for the corresponding the event boundary. REVECA uses frame position embedding to incorporate information before and after the event boundary. Furthermore, it employs features extracted using the temporal segment network and temporal-based pairwise difference method to learn temporal information. A semantic segmentation mask for the attentional pooling process is adopted to learn the subject of an event. Finally, LoRA is applied to fine-tune the image encoder to enhance the learning efficiency. REVECA yielded an average score of 50.97 on the Kinetics-GEBC test data, which is an improvement of 10.17 over the baseline method. Our code is available in https://github.com/TooTouch/REVECA.

本文描述了一种用于长视频理解工作坊中的通用边界事件字幕挑战的方法。我们设计了一个视频事件字幕生成器的丰富编-解码器框架(REVECA)，该框架利用视频的时空信息来生成相应的事件边界字幕。REVECA利用帧位置嵌入来合并事件边界前后的信息。此外，它采用时间段网络和基于时间的成对差方法提取特征以学习时间信息。采用了注意池化过程的语义分割掩模来学习事件的主题。最后，应用了LoRA来微调图像编码器以提高学习效率。在Kinetics-GEBC测试数据上，REVECA的平均得分为50.97，比基准方法提高了10.17。

REVECA -- 基于丰富编解码器框架的视频事件字幕生成器