Videos are more informative than images because they capture the dynamics of
the scene. By representing motion in videos, we can capture dynamic activities.
In this work, we introduce GPT-4 generated motion descriptions that capture
fine-grained motion descriptions of activities and apply them to three action
datasets. We evaluated several video-text models on the task of retrieval of
motion descriptions. We found that they fall far behind human expert
performance on two action datasets, raising the question of whether video-text
models understand motion in videos. To address it, we introduce a method of
improving motion understanding in video-text models by utilizing motion
descriptions. This method proves to be effective on two action datasets for the
motion description retrieval task. The results draw attention to the need for
quality captions involving fine-grained motion information in existing datasets
and demonstrate the effectiveness of the proposed pipeline in understanding
fine-grained motion during video-text retrieval.

通过引入生成的 GPT-4 运动描述应用于三个行动数据集并在运动描述检索任务上评估几种视频 - 文本模型，本研究探讨了视频与图片的信息性差异，着重关注视频 - 文本模型对于视频中运动的理解以及需要在现有数据集中加入细致动作信息的问题，并证明了利用动作描述提高视频 - 文本模型对于细致动作的理解的方法的有效性。