Weakly-supervised action segmentation is a task of learning to partition a
long video into several action segments, where training videos are only
accompanied by transcripts (ordered list of actions). Most of existing methods
need to infer pseudo segmentation for training by serial alignment between all
frames and the transcript, which is time-consuming and hard to be parallelized
while training. In this work, we aim to escape from this inefficient alignment
with massive but redundant frames, and instead to directly localize a few
action transitions for pseudo segmentation generation, where a transition
refers to the change from an action segment to its next adjacent one in the
transcript. As the true transitions are submerged in noisy boundaries due to
intra-segment visual variation, we propose a novel Action-Transition-Aware
Boundary Alignment (ATBA) framework to efficiently and effectively filter out
noisy boundaries and detect transitions. In addition, to boost the semantic
learning in the case that noise is inevitably present in the pseudo
segmentation, we also introduce video-level losses to utilize the trusted
video-level supervision. Extensive experiments show the effectiveness of our
approach on both performance and training speed.

本研究提出了一种新的弱监督动作分割方法，通过直接定位几个动作转换来生成伪分割，并利用边界对齐和视频级别监督来有效地过滤噪声并检测转换。实验证明了这种方法在性能和训练速度上的有效性。

基于动作 - 过渡感知边界对齐的高效有效的弱监督动作分割

Efficient and Effective Weakly-Supervised Action Segmentation via  Action-Transition-Aware Boundary Alignment

Noisy labels, inevitably existing in pseudo segmentation labels generated
from weak object-level annotations, severely hampers model optimization for
semantic segmentation. Previous works often rely on massive hand-crafted losses
and carefully-tuned hyper-parameters to resist noise, suffering poor
generalization capability and high model complexity. Inspired by recent
advances in meta learning, we argue that rather than struggling to tolerate
noise hidden behind clean labels passively, a more feasible solution would be
to find out the noisy regions actively, so as to simply ignore them during
model optimization. With this in mind, this work presents a novel meta learning
based semantic segmentation method, MetaSeg, that comprises a primary
content-aware meta-net (CAM-Net) to sever as a noise indicator for an arbitrary
segmentation model counterpart. Specifically, CAM-Net learns to generate
pixel-wise weights to suppress noisy regions with incorrect pseudo labels while
highlighting clean ones by exploiting hybrid strengthened features from image
content, providing straightforward and reliable guidance for optimizing the
segmentation model. Moreover, to break the barrier of time-consuming training
when applying meta learning to common large segmentation models, we further
present a new decoupled training strategy that optimizes different model layers
in a divide-and-conquer manner. Extensive experiments on object, medical,
remote sensing and human segmentation shows that our method achieves superior
performance, approaching that of fully supervised settings, which paves a new
promising way for omni-supervised semantic segmentation.

通过使用元学习技术，本论文提出了一种基于 CAM-Net 的新型语义分割方法 MetaSeg，该方法通过生成像素权重来抑制带有错误伪标签的噪声区域，并利用图像内容的增强特征提供简单可靠的优化指导，从而实现了与完全监督模型接近的出色性能，为全方位语义分割开辟了新的有前途的途径。