State-of-the-art methods for self-supervised sequential action alignment rely
on deep networks that find correspondences across videos in time. They either
learn frame-to-frame mapping across sequences, which does not leverage temporal
information, or assume monotonic alignment between each video pair, which
ignores variations in the order of actions. As such, these methods are not able
to deal with common real-world scenarios that involve background frames or
videos that contain non-monotonic sequence of actions.
In this paper, we propose an approach to align sequential actions in the wild
that involve diverse temporal variations. To this end, we propose an approach
to enforce temporal priors on the optimal transport matrix, which leverages
temporal consistency, while allowing for variations in the order of actions.
Our model accounts for both monotonic and non-monotonic sequences and handles
background frames that should not be aligned. We demonstrate that our approach
consistently outperforms the state-of-the-art in self-supervised sequential
action representation learning on four different benchmark datasets.

该研究提出了一种方法来解决深度学习在处理视频时不能考虑时间顺序信息，或忽略动作顺序差异所带来的问题，通过优先考虑时间一致性（同时允许动作关系顺序差异），解决自我学习顺序动作表示学习的问题，并在多个数据集上取得最优效果。