We present EgoExo-Fitness, a new full-body action understanding dataset,
featuring fitness sequence videos recorded from synchronized egocentric and
fixed exocentric (third-person) cameras. Compared with existing full-body
action understanding datasets, EgoExo-Fitness not only contains videos from
first-person perspectives, but also provides rich annotations. Specifically,
two-level temporal boundaries are provided to localize single action videos
along with sub-steps of each action. More importantly, EgoExo-Fitness
introduces innovative annotations for interpretable action judgement--including
technical keypoint verification, natural language comments on action execution,
and action quality scores. Combining all of these, EgoExo-Fitness provides new
resources to study egocentric and exocentric full-body action understanding
across dimensions of "what", "when", and "how well". To facilitate research on
egocentric and exocentric full-body action understanding, we construct
benchmarks on a suite of tasks (i.e., action classification, action
localization, cross-view sequence verification, cross-view skill determination,
and a newly proposed task of guidance-based execution verification), together
with detailed analysis. Code and data will be available at
this https URL

我们呈现了 EgoExo-Fitness，这是一个全新的全身动作理解数据集，通过同步的自主和固定的第三人称摄像机记录了健身序列视频。与现有的全身动作理解数据集相比，EgoExo-Fitness 不仅包含第一人称视角的视频，还提供了丰富的注释，包括单个动作视频的两级时间界限以及每个动作的子步骤等。重要的是，EgoExo-Fitness 引入了新颖的注释，包括技术关键点验证、对动作执行的自然语言评论以及动作质量评分，为研究自主和固定视角下全身动作理解提供了新的资源。为了促进对自主和固定视角下全身动作理解的研究，我们构建了一套任务基准（即动作分类、动作定位、跨视图序列验证、跨视图技能判断和新提出的基于指导的执行验证任务），并进行了详细分析。代码和数据将在此 https URL 提供。