We introduce an approach for pre-training egocentric video models using
large-scale third-person video datasets. Learning from purely egocentric data
is limited by low dataset scale and diversity, while using purely exocentric
(third-person) data introduces a large domain mismatch. Our idea is to discover
latent signals in third-person video that are predictive of key
egocentric-specific properties. Incorporating these signals as knowledge
distillation losses during pre-training results in models that benefit from
both the scale and diversity of third-person video data, as well as
representations that capture salient egocentric properties. Our experiments
show that our Ego-Exo framework can be seamlessly integrated into standard
video models; it outperforms all baselines when fine-tuned for egocentric
activity recognition, achieving state-of-the-art results on Charades-Ego and
EPIC-Kitchens-100.

本文提出使用大规模第三人称视频数据集进行预训练的以自我为中心的视频模型方法，通过在第三人称视频中发现预测自我特定属性的潜在信号，并将其作为知识蒸馏损失融入模型预训练中，在精细调节进行自我中心的活动识别时表现出最佳表现，取得 Charades-Ego 和 EPIC-Kitchens-100 的最新成果。