Surgical context inference has recently garnered significant attention in
robot-assisted surgery as it can facilitate workflow analysis, skill
assessment, and error detection. However, runtime context inference is
challenging since it requires timely and accurate detection of the interactions
among the tools and objects in the surgical scene based on the segmentation of
video data. On the other hand, existing state-of-the-art video segmentation
methods are often biased against infrequent classes and fail to provide
temporal consistency for segmented masks. This can negatively impact the
context inference and accurate detection of critical states. In this study, we
propose a solution to these challenges using a Space Time Correspondence
Network (STCN). STCN is a memory network that performs binary segmentation and
minimizes the effects of class imbalance. The use of a memory bank in STCN
allows for the utilization of past image and segmentation information, thereby
ensuring consistency of the masks. Our experiments using the publicly available
JIGSAWS dataset demonstrate that STCN achieves superior segmentation
performance for objects that are difficult to segment, such as needle and
thread, and improves context inference compared to the state-of-the-art. We
also demonstrate that segmentation and context inference can be performed at
runtime without compromising performance.

通过使用时空对应网络 (STCN) 解决视频分割的类别偏差和临界状态的准确检测问题，在机器人辅助手术中实现了优异的分割性能和上下文推断。

运用记忆网络进行手术运行时环境推理的机器人场景分割

Robotic Scene Segmentation with Memory Network for Runtime Surgical  Context Inference

The need for automatic surgical skills assessment is increasing, especially
because manual feedback from senior surgeons observing junior surgeons is prone
to subjectivity and time consuming. Thus, automating surgical skills evaluation
is a very important step towards improving surgical practice. In this paper, we
designed a Convolutional Neural Network (CNN) to evaluate surgeon skills by
extracting patterns in the surgeon motions performed in robotic surgery. The
proposed method is validated on the JIGSAWS dataset and achieved very
competitive results with 100% accuracy on the suturing and needle passing
tasks. While we leveraged from the CNNs efficiency, we also managed to mitigate
its black-box effect using class activation map. This feature allows our method
to automatically highlight which parts of the surgical task influenced the
skill prediction and can be used to explain the classification and to provide
personalized feedback to the trainee.

本文通过设计一种卷积神经网络 (CNN) 来自动评估机器人手术中的外科医生技能，经过我们提出的方法在 JIGSAWS 数据集上得到了非常有竞争力的结果，并通过类激活图来解释各分类并提供个性化反馈