This paper focuses on activity retrieval from a video query in an imbalanced scenario. In current query-by-activity-video literature, a common assumption is that all activities have sufficient labelled examples when learning an embedding. This assumption does however practically not hold, as only a portion of activities have many examples, while other activities are only described by few examples. In this paper, we propose a visual-semantic embedding network that explicitly deals with the imbalanced scenario for activity retrieval. Our network contains two novel modules. The visual alignment module performs a global alignment between the input video and fixed-sized visual bank representations for all activities. The semantic module performs an alignment between the input video and fixed-sized semantic activity representations. By matching videos with both visual and semantic activity representations that are of equal size over all activities, we no longer ignore infrequent activities during retrieval. Experiments on a new imbalanced activity retrieval benchmark show the effectiveness of our approach for all types of activities.

本文针对非平衡场景中的视频查询中的活动检索进行研究，提出了一种视觉-语义嵌入网络，该网络包含两个新模块：视觉对齐模块通过全局对齐输入视频和所有活动的固定大小视觉库表示之间的关系，语义模块在输入视频和固定大小的语义活动表示之间进行对齐。通过匹配具有相等规模的视觉和语义活动表示的视频，我们不再在检索过程中忽略不常见的活动。实验结果表明，我们的方法在各类活动的非平衡活动检索基准上具有有效性。

野外活动视频查询