Procedural activity understanding requires perceiving human actions in terms
of a broader task, where multiple keysteps are performed in sequence across a
long video to reach a final goal state -- such as the steps of a recipe or a
DIY fix-it task. Prior work largely treats keystep recognition in isolation of
this broader structure, or else rigidly confines keysteps to align with a
predefined sequential script. We propose discovering a task graph automatically
from how-to videos to represent probabilistically how people tend to execute
keysteps, and then leverage this graph to regularize keystep recognition in
novel videos. On multiple datasets of real-world instructional videos, we show
the impact: more reliable zero-shot keystep localization and improved video
representation learning, exceeding the state of the art.

我们提出从视频中自动发现任务图表示人们如何执行关键步骤的概率，并利用该图来规范化新视频中的关键步骤识别，从而在多个真实世界教学视频数据集上展示了影响：更可靠的零样本关键步骤定位和改进的视频表示学习，超越了现有技术水平。