We introduce VISOR, a new dataset of pixel annotations and a benchmark suite
for segmenting hands and active objects in egocentric video. VISOR annotates
videos from EPIC-KITCHENS, which comes with a new set of challenges not
encountered in current video segmentation datasets. Specifically, we need to
ensure both short- and long-term consistency of pixel-level annotations as
objects undergo transformative interactions, e.g. an onion is peeled, diced and
cooked - where we aim to obtain accurate pixel-level annotations of the peel,
onion pieces, chopping board, knife, pan, as well as the acting hands. VISOR
introduces an annotation pipeline, AI-powered in parts, for scalability and
quality. In total, we publicly release 272K manual semantic masks of 257 object
classes, 9.9M interpolated dense masks, 67K hand-object relations, covering 36
hours of 179 untrimmed videos. Along with the annotations, we introduce three
challenges in video object segmentation, interaction understanding and
long-term reasoning.
For data, code and leaderboards: this http URL

本文介绍了 VISOR 数据集，它是一个像素注释的新数据集，也是用于在自我中心视频中分割手和活动对象的基准套件。VISOR 主要应对了从 EPIC-KITCHENS 视频中的新挑战，并引入了一个 AI 动力注释流程，以获得像素级注释的一致性。该数据集总共公开了 272K 带标注的语义掩模，9.9M 插值稠密掩模，67K 手 - 物体关系，并涵盖 36 小时的 179 个未修剪的视频。此外，我们还介绍了三项有关视频对象分割，交互理解和长期推理的挑战。