We investigate exocentric-to-egocentric cross-view translation, which aims to
generate a first-person (egocentric) view of an actor based on a video
recording that captures the actor from a third-person (exocentric) perspective.
To this end, we propose a generative framework called Exo2Ego that decouples
the translation process into two stages: high-level structure transformation,
which explicitly encourages cross-view correspondence between exocentric and
egocentric views, and a diffusion-based pixel-level hallucination, which
incorporates a hand layout prior to enhance the fidelity of the generated
egocentric view. To pave the way for future advancements in this field, we
curate a comprehensive exo-to-ego cross-view translation benchmark. It consists
of a diverse collection of synchronized ego-exo tabletop activity video pairs
sourced from three public datasets: H2O, Aria Pilot, and Assembly101. The
experimental results validate that Exo2Ego delivers photorealistic video
results with clear hand manipulation details and outperforms several baselines
in terms of both synthesis quality and generalization ability to new actions.

研究探讨外心 - 内心跨视角转换，提出一种名为 Exo2Ego 的生成框架，将转换过程分解为高层次结构转换和基于扩散的像素级幻觉，为未来进一步的发展提供了一个综合的外心 - 内心跨视角转换基准，并通过实验证实了 Exo2Ego 在生成仿真质量和泛化能力方面优于其他基准。