This paper addresses a new problem of weakly-supervised online action
segmentation in instructional videos. We present a framework to segment
streaming videos online at test time using Dynamic Programming and show its
advantages over greedy sliding window approach. We improve our framework by
introducing the Online-Offline Discrepancy Loss (OODL) to encourage the
segmentation results to have a higher temporal consistency. Furthermore, only
during training, we exploit frame-wise correspondence between multiple views as
supervision for training weakly-labeled instructional videos. In particular, we
investigate three different multi-view inference techniques to generate more
accurate frame-wise pseudo ground-truth with no additional annotation cost. We
present results and ablation studies on two benchmark multi-view datasets,
Breakfast and IKEA ASM. Experimental results show efficacy of the proposed
methods both qualitatively and quantitatively in two domains of cooking and
assembly.

本文提出了一种弱监督的在线动作分割框架，采用动态规划方法来在线分割流媒体视频，并通过引入 Online-Offline Discrepancy Loss（OODL）优化分割结果的时间一致性，同时仅在训练期间使用多视角帧级一致性作为弱标注来提高精度及减少标注成本，实验结果在烹饪和组装等两个领域的 Breakfast 和 IKEA ASM 数据集中均得到了验证。