Understanding comprehensive assembly knowledge from videos is critical for
futuristic ultra-intelligent industry. To enable technological breakthrough, we
present HA-ViD - the first human assembly video dataset that features
representative industrial assembly scenarios, natural procedural knowledge
acquisition process, and consistent human-robot shared annotations.
Specifically, HA-ViD captures diverse collaboration patterns of real-world
assembly, natural human behaviors and learning progression during assembly, and
granulate action annotations to subject, action verb, manipulated object,
target object, and tool. We provide 3222 multi-view, multi-modality videos
(each video contains one assembly task), 1.5M frames, 96K temporal labels and
2M spatial labels. We benchmark four foundational video understanding tasks:
action recognition, action segmentation, object detection and multi-object
tracking. Importantly, we analyze their performance for comprehending knowledge
in assembly progress, process efficiency, task collaboration, skill parameters
and human intention. Details of HA-ViD is available at:
this https URL

为了实现未来的超级智能工业，本论文提出了 HA-ViD—— 第一个可用于代表性工业组装场景、自然过程知识获取过程和一致的人机共享注释的人类组装视频数据集，主要解决组装过程中知识理解、流程效率、任务协作、技能参数和人类意图等方面的问题。