We are interested in anticipating as early as possible the target location of
a person's object manipulation action in a 3D workspace from egocentric vision.
It is important in fields like human-robot collaboration, but has not yet
received enough attention from vision and learning communities. To stimulate
more research on this challenging egocentric vision task, we propose a large
multimodality dataset of more than 1 million frames of RGB-D and IMU streams,
and provide evaluation metrics based on our high-quality 2D and 3D labels from
semi-automatic annotation. Meanwhile, we design baseline methods using
recurrent neural networks and conduct various ablation studies to validate
their effectiveness. Our results demonstrate that this new task is worthy of
further study by researchers in robotics, vision, and learning communities.

该研究旨在从自我中心视角预测人的物体操纵行为的目标位置。他们提出了一个包含 RGB-D 和 IMU 流的大型多模态数据集，并设计了使用递归神经网络的基线方法，并进行了各种消融研究以验证其有效性，从而证明该新任务值得进一步学习。