Real-world tasks consist of multiple inter-dependent subtasks (e.g., a dirty
pan needs to be washed before it can be used for cooking). In this work, we aim
to model the causal dependencies between such subtasks from instructional
videos describing the task. This is a challenging problem since complete
information about the world is often inaccessible from videos, which demands
robust learning mechanisms to understand the causal structure of events. We
present Multimodal Subtask Graph Generation (MSG2), an approach that constructs
a Subtask Graph defining the dependency between a task's subtasks relevant to a
task from noisy web videos. Graphs generated by our multimodal approach are
closer to human-annotated graphs compared to prior approaches. MSG2 further
performs the downstream task of next subtask prediction 85% and 30% more
accurately than recent video transformer models in the ProceL and CrossTask
datasets, respectively.

该研究旨在从任务的教学视频中建立多模态子任务图，以便更准确地预测下一个子任务，并且与先前方法相比，建立的图更接近于人工标注的图。