Action segmentation is the task of predicting the actions for each frame of a
video. As obtaining the full annotation of videos for action segmentation is
expensive, weakly supervised approaches that can learn only from transcripts
are appealing. In this paper, we propose a novel end-to-end approach for weakly
supervised action segmentation based on a two-branch neural network. The two
branches of our network predict two redundant but different representations for
action segmentation and we propose a novel mutual consistency (MuCon) loss that
enforces the consistency of the two redundant representations. Using the MuCon
loss together with a loss for transcript prediction, our proposed approach
achieves the accuracy of state-of-the-art approaches while being $14$ times
faster to train and $20$ times faster during inference. The MuCon loss proves
beneficial even in the fully supervised setting.

本文提出一种基于双分支神经网络的弱监督动作分割方法，并提出了一种新的相互一致性损失（MuCon）来保证分割的一致性，结合文本预测损失达到了当前最新技术方法的准确率，且训练时间快了 14 倍，推理时间快了 20 倍，同时证明了 MuCon 损失在全监督设置下也有益处。