Video Correlation Learning (VCL), which aims to analyze the relationships
between videos, has been widely studied and applied in various general video
tasks. However, applying VCL to instructional videos is still quite challenging
due to their intrinsic procedural temporal structure. Specifically, procedural
knowledge is critical for accurate correlation analyses on instructional
videos. Nevertheless, current procedure-learning methods heavily rely on
step-level annotations, which are costly and not scalable. To address this
problem, we introduce a weakly supervised framework called Collaborative
Procedure Alignment (CPA) for procedure-aware correlation learning on
instructional videos. Our framework comprises two core modules: collaborative
step mining and frame-to-step alignment. The collaborative step mining module
enables simultaneous and consistent step segmentation for paired videos,
leveraging the semantic and temporal similarity between frames. Based on the
identified steps, the frame-to-step alignment module performs alignment between
the frames and steps across videos. The alignment result serves as a
measurement of the correlation distance between two videos. We instantiate our
framework in two distinct instructional video tasks: sequence verification and
action quality assessment. Extensive experiments validate the effectiveness of
our approach in providing accurate and interpretable correlation analyses for
instructional videos.

我们介绍了一种用于指导性视频的程序感知相关性学习的弱监督框架，该框架包含协同步骤挖掘和帧到步骤对齐两个核心模块，并在序列验证和行动质量评估两个任务中进行了实例化，实验证实了我们的方法在为指导性视频提供准确和可解释的相关性分析方面的有效性。