The ability to locate and classify action segments in long untrimmed video is
of particular interest to many applications such as autonomous cars, robotics
and healthcare applications. Today, the most popular pipeline for action
segmentation is composed of encoding the frames into feature vectors, which are
then processed by a temporal model for segmentation. In this paper we present a
self-supervised method that comes in the middle of the standard pipeline and
generated refined representations of the original feature vectors. Experiments
show that this method improves the performance of existing models on different
sub-tasks of action segmentation, even without additional hyper parameter
tuning.

通过自我监督方法生成原始特征向量的改进表示，提高了现有模型在行动分割的不同子任务上的性能。

SFGANS 自监督的人体动作分割未来生成器

SFGANS Self-supervised Future Generator for human ActioN Segmentation

Since its introduction in 2018, EPIC-KITCHENS has attracted attention as the
largest egocentric video benchmark, offering a unique viewpoint on people's
interaction with objects, their attention, and even intention. In this paper,
we detail how this large-scale dataset was captured by 32 participants in their
native kitchen environments, and densely annotated with actions and object
interactions. Our videos depict nonscripted daily activities, as recording is
started every time a participant entered their kitchen. Recording took place in
4 countries by participants belonging to 10 different nationalities, resulting
in highly diverse kitchen habits and cooking styles. Our dataset features 55
hours of video consisting of 11.5M frames, which we densely labelled for a
total of 39.6K action segments and 454.2K object bounding boxes. Our annotation
is unique in that we had the participants narrate their own videos after
recording, thus reflecting true intention, and we crowd-sourced ground-truths
based on these. We describe our object, action and. anticipation challenges,
and evaluate several baselines over two test splits, seen and unseen kitchens.
We introduce new baselines that highlight the multimodal nature of the dataset
and the importance of explicit temporal modelling to discriminate fine-grained
actions e.g. 'closing a tap' from 'opening' it up.

本文介绍了 EPIC-KITCHENS 数据集的构建，该数据集具有大规模、种类多样的人们在厨房的日常行为记录，并且由多名来自不同国籍的参与者拍摄，其中所有视频都进行了密集注释和真实意图的讲述，并使用了多模态建模等基础线和新基础线进行了评估。

EPIC-KITCHENS 数据集：收集、挑战和基准

The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines

First-person vision is gaining interest as it offers a unique viewpoint on
people's interaction with objects, their attention, and even intention.
However, progress in this challenging domain has been relatively slow due to
the lack of sufficiently large datasets. In this paper, we introduce
EPIC-KITCHENS, a large-scale egocentric video benchmark recorded by 32
participants in their native kitchen environments. Our videos depict
nonscripted daily activities: we simply asked each participant to start
recording every time they entered their kitchen. Recording took place in 4
cities (in North America and Europe) by participants belonging to 10 different
nationalities, resulting in highly diverse cooking styles. Our dataset features
55 hours of video consisting of 11.5M frames, which we densely labeled for a
total of 39.6K action segments and 454.3K object bounding boxes. Our annotation
is unique in that we had the participants narrate their own videos (after
recording), thus reflecting true intention, and we crowd-sourced ground-truths
based on these. We describe our object, action and anticipation challenges, and
evaluate several baselines over two test splits, seen and unseen kitchens.
Dataset and Project page: this http URL

本文介绍了 EPIC-KITCHENS 数据库，它是一个大规模的第一人称视角视频数据库，其中包含了来自 32 个参与者的 55 小时视频，多样性与大规模性深度融合，视频被密集标记为总共 39.6K 个动作段和 454.3K 个物体边界框，并且注释与真实意图紧密相关，该数据库可用于第一人称视角视频领域的挑战评估和研究。