This work demonstrates the implementation and use of an encoder-decoder model to perform a many-to-many mapping of video data to text captions. The many-to-many mapping occurs via an input temporal sequence of video frames to an output sequence of words to form a caption sentence. Data preprocessing, model construction, and model training are discussed. Caption correctness is evaluated using 2-gram BLEU scores across the different splits of the dataset. Specific examples of output captions were shown to demonstrate model generality over the video temporal dimension. Predicted captions were shown to generalize over video action, even in instances where the video scene changed dramatically. Model architecture changes are discussed to improve sentence grammar and correctness.

本文展示了使用编码-解码模型来实现视频数据到文本字幕的多对多映射的方法，并讨论了数据预处理、模型构建和训练。通过对数据集的不同分割使用2元BLEU分数来评估字幕的准确性。通过特定的输出示例字幕表明模型在视频时间维度上具有通用性。预测的字幕表明在视频动作方面具有普适性，即使在视频场景发生剧烈变化的情况下。还讨论了改进句子语法和准确性的模型架构变化。

基于编码器-解码器的长短期记忆（LSTM）视频字幕模型