Pre-training vision-language representations on human action videos has emerged as a promising approach to reduce reliance on large-scale expert demonstrations for training embodied agents. However, prior methods often employ time contrastive learning based on goal-reaching heuristics, progressively aligning language instructions from the initial to the final frame. This overemphasis on future frames can result in erroneous vision-language associations, as actions may terminate early or include irrelevant moments in the end. To address this issue, we propose Action Temporal Coherence Learning (AcTOL) to learn ordered and continuous vision-language representations without rigid goal-based constraint. AcTOL treats a video as a continuous trajectory where it (1) contrasts semantic differences between frames to reflect their natural ordering, and (2) imposes a local Brownian bridge constraint to ensure smooth transitions across intermediate frames. Extensive imitation learning experiments across varying numbers of demonstrations show that the pretrained features significantly enhance downstream manipulation tasks by up to 49% with high robustness to different linguistic styles of instructions, offering a viable pathway toward generalized embodied agents. The source code is included in the supplementary material for reference.

本研究针对现有视觉-语言预训练方法中因过度关注未来帧而导致的视觉-语言关联错误问题，提出了一种新颖的动作时间连贯性学习方法（AcTOL）。这一方法通过对视频帧的自然顺序进行语义对比学习，同时实现了平滑过渡，显著提升了下游操作任务的性能，增强了对不同语言风格指令的鲁棒性，推动了可泛化的具身智能体的发展。

可证明的视觉-语言预训练中的顺序性和连续性，用于可泛化的具身智能体