Promptly identifying procedural errors from egocentric videos in an online
setting is highly challenging and valuable for detecting mistakes as soon as
they happen. This capability has a wide range of applications across various
fields, such as manufacturing and healthcare. The nature of procedural mistakes
is open-set since novel types of failures might occur, which calls for
one-class classifiers trained on correctly executed procedures. However, no
technique can currently detect open-set procedural mistakes online. We propose
PREGO, the first online one-class classification model for mistake detection in
PRocedural EGOcentric videos. PREGO is based on an online action recognition
component to model the current action, and a symbolic reasoning module to
predict the next actions. Mistake detection is performed by comparing the
recognized current action with the expected future one. We evaluate PREGO on
two procedural egocentric video datasets, Assembly101 and Epic-tent, which we
adapt for online benchmarking of procedural mistake detection to establish
suitable benchmarks, thus defining the Assembly101-O and Epic-tent-O datasets,
respectively.

通过在线的一类分类模型 PREGO 在过程自我中心视频中检测错误，从而准确识别程序性错误，该模型基于在线动作识别和符号推理模块实现错误检测。

PREGO：PRocedural EGOcentric 视频中的在线错误检测

PREGO: online mistake detection in PRocedural EGOcentric videos

Although action recognition for procedural tasks has received notable
attention, it has a fundamental flaw in that no measure of success for actions
is provided. This limits the applicability of such systems especially within
the industrial domain, since the outcome of procedural actions is often
significantly more important than the mere execution. To address this
limitation, we define the novel task of procedure step recognition (PSR),
focusing on recognizing the correct completion and order of procedural steps.
Alongside the new task, we also present the multi-modal IndustReal dataset.
Unlike currently available datasets, IndustReal contains procedural errors
(such as omissions) as well as execution errors. A significant part of these
errors are exclusively present in the validation and test sets, making
IndustReal suitable to evaluate robustness of algorithms to new, unseen
mistakes. Additionally, to encourage reproducibility and allow for scalable
approaches trained on synthetic data, the 3D models of all parts are publicly
available. Annotations and benchmark performance are provided for action
recognition and assembly state detection, as well as the new PSR task.
IndustReal, along with the code and model weights, is available at:
this https URL .

该研究提出了一种新的任务，即过程步骤识别（PSR），旨在识别程序步骤的正确完成和顺序。研究还介绍了多模态数据集 IndustReal，该数据集包含程序错误和执行错误，并提供用于行动识别和装配状态检测的注释和基准性能。