Egocentric vision consists in acquiring images along the day from a first
person point-of-view using wearable cameras. The automatic analysis of this
information allows to discover daily patterns for improving the quality of life
of the user. A natural topic that arises in egocentric vision is storytelling,
that is, how to understand and tell the story relying behind the pictures. In
this paper, we tackle storytelling as an egocentric sequences description
problem. We propose a novel methodology that exploits information from
temporally neighboring events, matching precisely the nature of egocentric
sequences. Furthermore, we present a new method for multimodal data fusion
consisting on a multi-input attention recurrent network. We also publish the
first dataset for egocentric image sequences description, consisting of 1,339
events with 3,991 descriptions, from 55 days acquired by 11 people.
Furthermore, we prove that our proposal outperforms classical attentional
encoder-decoder methods for video description.

本研究旨在利用一种新的方法，即多态输入的注意力循环网络，来描述自我中心影像序列的故事情节，并发布了第一个自我中心影像序列描述数据集，该方法的表现优于传统的注意力编码器 - 解码器方法。