Egocentric videos offer fine-grained information for high-fidelity modeling of human behaviors. Hands and interacting objects are one crucial aspect of understanding a viewer's behaviors and intentions. We provide a labeled dataset consisting of 11,243 egocentric images with per-pixel segmentation labels of hands and objects being interacted with during a diverse array of daily activities. Our dataset is the first to label detailed hand-object contact boundaries. We introduce a context-aware compositional data augmentation technique to adapt to out-of-distribution YouTube egocentric video. We show that our robust hand-object segmentation model and dataset can serve as a foundational tool to boost or enable several downstream vision applications, including hand state classification, video activity recognition, 3D mesh reconstruction of hand-object interactions, and video inpainting of hand-object foregrounds in egocentric videos. Dataset and code are available at: https://github.com/owenzlz/EgoHOS

本文提供了一种具有像素级细分标签的egocentric图像数据集，其中包含11,243个日常活动中与对象相互作用时人手和对象的标签数据。我们引入了上下文感知的组合数据增强技术以适应分布外的YouTube egocentric视频，并展示了我们坚韧的手物分割模型和数据集可以作为基础工具，促进或实现手势状态分类、视频活动识别、手物交互的3D网格重建，以及对egocentric视频中的手物前景进行视频修复。

细粒度自我中心手部-物体分割：数据集、模型和应用