Being able to map the activities of others into one's own point of view is
one fundamental human skill even from a very early age. Taking a step toward
understanding this human ability, we introduce EgoExoLearn, a large-scale
dataset that emulates the human demonstration following process, in which
individuals record egocentric videos as they execute tasks guided by
demonstration videos. Focusing on the potential applications in daily
assistance and professional support, EgoExoLearn contains egocentric and
demonstration video data spanning 120 hours captured in daily life scenarios
and specialized laboratories. Along with the videos we record high-quality gaze
data and provide detailed multimodal annotations, formulating a playground for
modeling the human ability to bridge asynchronous procedural actions from
different viewpoints. To this end, we present benchmarks such as cross-view
association, cross-view action planning, and cross-view referenced skill
assessment, along with detailed analysis. We expect EgoExoLearn can serve as an
important resource for bridging the actions across views, thus paving the way
for creating AI agents capable of seamlessly learning by observing humans in
the real world. Code and data can be found at:
this https URL

通过记录日常生活中的主体视角视频和示范视频，并提供高质量凝视数据和详细的多模态注释，EgoExoLearn 数据集旨在模拟人类桥接不同视角中的异步过程性动作的能力，并提供了跨视角关联、跨视角行动规划和跨视角技能评估等基准，为观察现实世界中的人类而无缝学习的 AI 代理创造了可能性。

EgoExoLearn：实现现实世界异步自我和外部中心的程序化活动的数据集

EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric  View of Procedural Activities in Real World

Estimating 3D poses of multiple humans in real-time is a classic but still
challenging task in computer vision. Its major difficulty lies in the ambiguity
in cross-view association of 2D poses and the huge state space when there are
multiple people in multiple views. In this paper, we present a novel solution
for multi-human 3D pose estimation from multiple calibrated camera views. It
takes 2D poses in different camera coordinates as inputs and aims for the
accurate 3D poses in the global coordinate. Unlike previous methods that
associate 2D poses among all pairs of views from scratch at every frame, we
exploit the temporal consistency in videos to match the 2D inputs with 3D poses
directly in 3-space. More specifically, we propose to retain the 3D pose for
each person and update them iteratively via the cross-view multi-human
tracking. This novel formulation improves both accuracy and efficiency, as we
demonstrated on widely-used public datasets. To further verify the scalability
of our method, we propose a new large-scale multi-human dataset with 12 to 28
camera views. Without bells and whistles, our solution achieves 154 FPS on 12
cameras and 34 FPS on 28 cameras, indicating its ability to handle large-scale
real-world applications. The proposed dataset is released at
this https URL

本论文提出了一种基于多个校准相机视角的、面向实时应用的多人三维姿态估计解决方案，利用视频中的时间一致性直接在三维空间中匹配二维输入与三维姿态，通过跨视图多人跟踪迭代地更新姿态，从而提高了准确性和效率，并介绍了新的大规模多人数据集。