Video learning is an important task in computer vision and has experienced
increasing interest over the recent years. Since even a small amount of videos
easily comprises several million frames, methods that do not rely on a
frame-level annotation are of special importance. In this work, we propose a
novel learning algorithm with a Viterbi-based loss that allows for online and
incremental learning of weakly annotated video data. We moreover show that
explicit context and length modeling leads to huge improvements in video
segmentation and labeling tasks andinclude these models into our framework. On
several action segmentation benchmarks, we obtain an improvement of up to 10%
compared to current state-of-the-art methods.

本文提出了一种基于 Viterbi 损失的学习算法，能够在线和逐步地学习弱标注的视频数据，并展示了显式的上下文和长度建模对于视频分割和标记任务的巨大改进，并将这些模型纳入我们的框架之中，在几个动作分割基准测试中，与当前最先进的方法相比，我们实现了高达 10% 的改进。

神经网络维特比算法：弱监督视频学习框架

NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning

Action segmentation as a milestone towards building automatic systems to
understand untrimmed videos has received considerable attention in the recent
years. It is typically being modeled as a sequence labeling problem but
contains intrinsic and sufficient differences than text parsing or speech
processing. In this paper, we introduce a novel hybrid temporal convolutional
and recurrent network (TricorNet), which has an encoder-decoder architecture:
the encoder consists of a hierarchy of temporal convolutional kernels that
capture the local motion changes of different actions; the decoder is a
hierarchy of recurrent neural networks that are able to learn and memorize
long-term action dependencies after the encoding stage. Our model is simple but
extremely effective in terms of video sequence labeling. The experimental
results on three public action segmentation datasets have shown that the
proposed model achieves superior performance over the state of the art.

本文提出了一种新的混合时态卷积和循环神经网络 (TricorNet) 模型，其具有编码器 - 解码器结构，通过编码器捕捉不同动作的局部运动变化，解码器能够学习和记忆长期的动作依赖性，该模型在三个行动分割数据集上的实验结果表明，在视频序列标注方面，所提出的模型优于现有技术水平。

TricorNet：用于视频动作分割的混合时态卷积和循环网络

TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for  Video Action Segmentation

Joint segmentation and classification of fine-grained actions is important
for applications of human-robot interaction, video surveillance, and human
skill evaluation. However, despite substantial recent progress in large-scale
action classification, the performance of state-of-the-art fine-grained action
recognition approaches remains low. We propose a model for action segmentation
which combines low-level spatiotemporal features with a high-level segmental
classifier. Our spatiotemporal CNN is comprised of a spatial component that
uses convolutional filters to capture information about objects and their
relationships, and a temporal component that uses large 1D convolutional
filters to capture information about how object relationships change across
time. These features are used in tandem with a semi-Markov model that models
transitions from one action to another. We introduce an efficient constrained
segmental inference algorithm for this model that is orders of magnitude faster
than the current approach. We highlight the effectiveness of our Segmental
Spatiotemporal CNN on cooking and surgical action datasets for which we observe
substantially improved performance relative to recent baseline methods.

提出了一个基于低级时空特征和高级的段分类器的动作分割模型， 结合了半马尔可夫模型， 并引入一种比当前方法快数倍的有效的有约束的段推断算法。在烹饪和手术动作数据集上观察到明显的改进。