This paper aims for event recognition when video examples are scarce or even
completely absent. The key in such a challenging setting is a semantic video
representation. Rather than building the representation from individual
attribute detectors and their annotations, we propose to learn the entire
representation from freely available web videos and their descriptions using an
embedding between video features and term vectors. In our proposed embedding,
which we call VideoStory, the correlations between the terms are utilized to
learn a more effective representation by optimizing a joint objective balancing
descriptiveness and predictability.We show how learning the VideoStory using a
multimodal predictability loss, including appearance, motion and audio
features, results in a better predictable representation. We also propose a
variant of VideoStory to recognize an event in video from just the important
terms in a text query by introducing a term sensitive descriptiveness loss. Our
experiments on three challenging collections of web videos from the NIST
TRECVID Multimedia Event Detection and Columbia Consumer Videos datasets
demonstrate: i) the advantages of VideoStory over representations using
attributes or alternative embeddings, ii) the benefit of fusing video
modalities by an embedding over common strategies, iii) the complementarity of
term sensitive descriptiveness and multimodal predictability for event
recognition without examples. By it abilities to improve predictability upon
any underlying video feature while at the same time maximizing semantic
descriptiveness, VideoStory leads to state-of-the-art accuracy for both few-
and zero-example recognition of events in video.

本文提出了一种叫做 VideoStory 的语义视频表示方法，通过学习来自网络视频和其描述的嵌入来构建整个表示，同时通过多模态可预测性损失来提高描述性和可预测性，并利用术语敏感的描述性损失来识别没有例子的视频事件，从而在视频事件的少量和没有例子的识别方面提高了准确性。