With the increasing availability of wearable devices, research on egocentric
activity recognition has received much attention recently. In this paper, we
build a Multimodal Egocentric Activity dataset which includes egocentric videos
and sensor data of 20 fine-grained and diverse activity categories. We present
a novel strategy to extract temporal trajectory-like features from sensor data.
We propose to apply the Fisher Kernel framework to fuse video and temporal
enhanced sensor features. Experiment results show that with careful design of
feature extraction and fusion algorithm, sensor data can enhance
information-rich video data. We make publicly available the Multimodal
Egocentric Activity dataset to facilitate future research.

本文介绍了一个基于 Wearable 设备的多模态自我中心活动识别数据集，其中提出了一种从传感器数据中提取时空轨迹特征的新颖策略，并应用 Fisher Kernel 框架融合视频和时空增强的传感器特征，实验结果表明，精心设计的特征提取和融合算法可以提高信息丰富的视频数据的质量。

多模态 Fisher 向量的自我中心活动识别

Egocentric Activity Recognition with Multimodal Fisher Vector

Compared to image representation based on low-level local descriptors, deep
neural activations of Convolutional Neural Networks (CNNs) are richer in
mid-level representation, but poorer in geometric invariance properties. In
this paper, we present a straightforward framework for better image
representation by combining the two approaches. To take advantages of both
representations, we propose an efficient method to extract a fair amount of
multi-scale dense local activations from a pre-trained CNN. We then aggregate
the activations by Fisher kernel framework, which has been modified with a
simple scale-wise normalization essential to make it suitable for CNN
activations. Replacing the direct use of a single activation vector with our
representation demonstrates significant performance improvements: +17.76 (Acc.)
on MIT Indoor 67 and +7.18 (mAP) on PASCAL VOC 2007. The results suggest that
our proposal can be used as a primary image representation for better
performances in visual recognition tasks.

本文介绍了一种将传统的基于低级局部描述符的图像表示法和卷积神经网络的深度神经激活相结合的框架，提出了一种从预训练的 CNN 中提取公平数量的多尺度密集局部激活的有效方法，并通过改进的 Fisher kernel 框架聚合这些激活。实验证明，该方法可用作更好地执行视觉识别任务的一种主要图像表示。