Robot learning of manipulation skills is hindered by the scarcity of diverse,
unbiased datasets. While curated datasets can help, challenges remain in
generalizability and real-world transfer. Meanwhile, large-scale "in-the-wild"
video datasets have driven progress in computer vision through self-supervised
techniques. Translating this to robotics, recent works have explored learning
manipulation skills by passively watching abundant videos sourced online.
Showing promising results, such video-based learning paradigms provide scalable
supervision while reducing dataset bias. This survey reviews foundations such
as video feature representation learning techniques, object affordance
understanding, 3D hand/body modeling, and large-scale robot resources, as well
as emerging techniques for acquiring robot manipulation skills from
uncontrolled video demonstrations. We discuss how learning only from observing
large-scale human videos can enhance generalization and sample efficiency for
robotic manipulation. The survey summarizes video-based learning approaches,
analyses their benefits over standard datasets, survey metrics, and benchmarks,
and discusses open challenges and future directions in this nascent domain at
the intersection of computer vision, natural language processing, and robot
learning.

机器人通过观察大规模人类视频学习可以增强对机器人操纵的泛化性和样本效率。