Most benchmarks for studying surgical interventions focus on a specific
challenge instead of leveraging the intrinsic complementarity among different
tasks. In this work, we present a new experimental framework towards holistic
surgical scene understanding. First, we introduce the Phase, Step, Instrument,
and Atomic Visual Action recognition (PSI-AVA) Dataset. PSI-AVA includes
annotations for both long-term (Phase and Step recognition) and short-term
reasoning (Instrument detection and novel Atomic Action recognition) in
robot-assisted radical prostatectomy videos. Second, we present Transformers
for Action, Phase, Instrument, and steps Recognition (TAPIR) as a strong
baseline for surgical scene understanding. TAPIR leverages our dataset's
multi-level annotations as it benefits from the learned representation on the
instrument detection task to improve its classification capacity. Our
experimental results in both PSI-AVA and other publicly available databases
demonstrate the adequacy of our framework to spur future research on holistic
surgical scene understanding.

本文提出了一种新的基于综合外科场景理解的实验框架，通过介绍 PSI-AVA 数据集和 TAPIR 模型，展示了这种框架和其他公共数据库上实验结果的优越性和多级注释的好处。