With the success of deep learning in classifying short trimmed videos, more
attention has been focused on temporally segmenting and classifying activities
in long untrimmed videos. State-of-the-art approaches for action segmentation
utilize several layers of temporal convolution and temporal pooling. Despite
the capabilities of these approaches in capturing temporal dependencies, their
predictions suffer from over-segmentation errors. In this paper, we propose a
multi-stage architecture for the temporal action segmentation task that
overcomes the limitations of the previous approaches. The first stage generates
an initial prediction that is refined by the next ones. In each stage we stack
several layers of dilated temporal convolutions covering a large receptive
field with few parameters. While this architecture already performs well, lower
layers still suffer from a small receptive field. To address this limitation,
we propose a dual dilated layer that combines both large and small receptive
fields. We further decouple the design of the first stage from the refining
stages to address the different requirements of these stages. Extensive
evaluation shows the effectiveness of the proposed model in capturing
long-range dependencies and recognizing action segments. Our models achieve
state-of-the-art results on three datasets: 50Salads, Georgia Tech Egocentric
Activities (GTEA), and the Breakfast dataset.

本文提出了一种多阶段架构的深度学习模型，通过利用扩张时间卷积和双重扩张时间卷积层，实现了在长时间的未修剪视频中对动作进行准确的时序分割，并在三个数据集上取得了最先进的结果。

MS-TCN++：基于多阶段时间卷积网络的动作分割

MS-TCN++: Multi-Stage Temporal Convolutional Network for Action  Segmentation

Temporally locating and classifying action segments in long untrimmed videos
is of particular interest to many applications like surveillance and robotics.
While traditional approaches follow a two-step pipeline, by generating
frame-wise probabilities and then feeding them to high-level temporal models,
recent approaches use temporal convolutions to directly classify the video
frames. In this paper, we introduce a multi-stage architecture for the temporal
action segmentation task. Each stage features a set of dilated temporal
convolutions to generate an initial prediction that is refined by the next one.
This architecture is trained using a combination of a classification loss and a
proposed smoothing loss that penalizes over-segmentation errors. Extensive
evaluation shows the effectiveness of the proposed model in capturing
long-range dependencies and recognizing action segments. Our model achieves
state-of-the-art results on three challenging datasets: 50Salads, Georgia Tech
Egocentric Activities (GTEA), and the Breakfast dataset.

本研究提出了一种多阶段架构方法来解决视频动作分段问题，使用扩张卷积神经网络来生成初始预测并利用平滑损失进行训练，模型在多个数据集上实现了最新的研究成果。