Modelling interactions between humans and objects in natural environments is
central to many applications including gaming, virtual and mixed reality, as
well as human behavior analysis and human-robot collaboration. This challenging
operation scenario requires generalization to vast number of objects, scenes,
and human actions. Unfortunately, there exist no such dataset. Moreover, this
data needs to be acquired in diverse natural environments, which rules out 4D
scanners and marker based capture systems. We present BEHAVE dataset, the first
full body human- object interaction dataset with multi-view RGBD frames and
corresponding 3D SMPL and object fits along with the annotated contacts between
them. We record around 15k frames at 5 locations with 8 subjects performing a
wide range of interactions with 20 common objects. We use this data to learn a
model that can jointly track humans and objects in natural environments with an
easy-to-use portable multi-camera setup. Our key insight is to predict
correspondences from the human and the object to a statistical body model to
obtain human-object contacts during interactions. Our approach can record and
track not just the humans and objects but also their interactions, modeled as
surface contacts, in 3D. Our code and data can be found at:
this http URL

BEHAVE 数据集记录了 8 名主体在 5 个位置进行与 20 种常见对象的广泛互动，利用多视角 RGBD 帧和相应的 3D SMPL 和对象适配以及它们之间注释的接触，我们使用此数据学习模型，能够共同跟踪人类和物体的互动，同时可以在 3D 中对其交互进行建模，我们的关键洞察是预测与体统计模型中的人类和物品的对应关系，以获得互动期间的人 - 物接触。