Automatic action identification from video and kinematic data is an important
machine learning problem with applications ranging from robotics to smart
health. Most existing works focus on identifying coarse actions such as
running, climbing, or cutting a vegetable, which have relatively long
durations. This is an important limitation for applications that require the
identification of subtle motions at high temporal resolution. For example, in
stroke recovery, quantifying rehabilitation dose requires differentiating
motions with sub-second durations. Our goal is to bridge this gap. To this end,
we introduce a large-scale, multimodal dataset, StrokeRehab, as a new
action-recognition benchmark that includes subtle short-duration actions
labeled at a high temporal resolution. These short-duration actions are called
functional primitives, and consist of reaches, transports, repositions,
stabilizations, and idles. The dataset consists of high-quality Inertial
Measurement Unit sensors and video data of 41 stroke-impaired patients
performing activities of daily living like feeding, brushing teeth, etc. We
show that current state-of-the-art models based on segmentation produce noisy
predictions when applied to these data, which often leads to overcounting of
actions. To address this, we propose a novel approach for high-resolution
action identification, inspired by speech-recognition techniques, which is
based on a sequence-to-sequence model that directly predicts the sequence of
actions. This approach outperforms current state-of-the-art methods on the
StrokeRehab dataset, as well as on the standard benchmark datasets 50Salads,
Breakfast, and Jigsaws.

本文介绍了一个新数据集 StrokeRehab，旨在解决机器学习中的一个重要问题，即从视频和运动数据中识别出细微的运动，提出了一种以语音识别技术为灵感的基于序列到序列模型的高分辨率动作识别方法，并在 StrokeRehab 数据集以及标准测试数据集 50Salads，Breakfast 和 Jigsaws 上取得了当前最优的结果。