Given video demonstrations and paired narrations of an at-home procedural
task such as changing a tire, we present an approach to extract the underlying
task structure -- relevant actions and their temporal dependencies -- via
action-centric task graphs. Learnt structured representations from our method,
Action Dynamics Task Graphs (ADTG), can then be used for understanding such
tasks in unseen videos of humans performing them. Furthermore, ADTG can enable
providing user-centric guidance to humans in these tasks, either for performing
them better or for learning new tasks. Specifically, we show how ADTG can be
used for: (1) tracking an ongoing task, (2) recommending next actions, and (3)
planning a sequence of actions to accomplish a procedural task. We compare
against state-of-the-art Neural Task Graph method and demonstrate substantial
gains on 18 procedural tasks from the CrossTask dataset, including 30.1%
improvement in task tracking accuracy and 20.3% accuracy gain in next action
prediction.

本文通过 Action Dynamics Task Graphs (ATDG) 提取任务的结构 - 相关动作及其时间依赖性来理解人类执行任务的未见视频。ATDG 还可以用于提供用户中心指导，包括跟踪正在执行中的任务、推荐下一步行动和规划动作序列，相较于当前最新的神经任务图法，在 CrossTask 数据集的 18 项操作中，改进了 30.1％的任务跟踪准确性和 20.3％的下一步行动预测准确度。