Purpose: Advances in surgical phase recognition are generally led by training
deeper networks. Rather than going further with a more complex solution, we
believe that current models can be exploited better. We propose a
self-knowledge distillation framework that can be integrated into current
state-of-the-art (SOTA) models without requiring any extra complexity to the
models or annotations.
Methods: Knowledge distillation is a framework for network regularization
where knowledge is distilled from a teacher network to a student network. In
self-knowledge distillation, the student model becomes the teacher such that
the network learns from itself. Most phase recognition models follow an
encoder-decoder framework. Our framework utilizes self-knowledge distillation
in both stages. The teacher model guides the training process of the student
model to extract enhanced feature representations from the encoder and build a
more robust temporal decoder to tackle the over-segmentation problem.
Results: We validate our proposed framework on the public dataset Cholec80.
Our framework is embedded on top of four popular SOTA approaches and
consistently improves their performance. Specifically, our best GRU model
boosts performance by +3.33% accuracy and +3.95% F1-score over the same
baseline model.
Conclusion: We embed a self-knowledge distillation framework for the first
time in the surgical phase recognition training pipeline. Experimental results
demonstrate that our simple yet powerful framework can improve performance of
existing phase recognition models. Moreover, our extensive experiments show
that even with 75% of the training set we still achieve performance on par with
the same baseline model trained on the full set.

本文通过引入自我知识蒸馏框架来改善手术阶段识别模型，实验结果表明，基于这一框架的模型在 4 种流行的 state-of-the-art 方法上的准确度和 F1-score 分别提高了 3.33％和 3.95％，并且即使使用 75％的训练集，其性能仍与完整训练集上基线模型的性能相当。

自我知识蒸馏用于外科手术阶段识别

Self-Knowledge Distillation for Surgical Phase Recognition

Phase recognition plays an essential role for surgical workflow analysis in
computer assisted intervention. Transformer, originally proposed for sequential
data modeling in natural language processing, has been successfully applied to
surgical phase recognition. Existing works based on transformer mainly focus on
modeling attention dependency, without introducing auto-regression. In this
work, an Auto-Regressive Surgical Transformer, referred as ARST, is first
proposed for on-line surgical phase recognition from laparoscopic videos,
modeling the inter-phase correlation implicitly by conditional probability
distribution. To reduce inference bias and to enhance phase consistency, we
further develop a consistency constraint inference strategy based on
auto-regression. We conduct comprehensive validations on a well-known public
dataset Cholec80. Experimental results show that our method outperforms the
state-of-the-art methods both quantitatively and qualitatively, and achieves an
inference rate of 66 frames per second (fps).

本文提出了一种名为 ARST 的自回归手术变压器，用于从腹腔镜视频中实时识别手术阶段。通过条件概率分布隐式地建模了阶段间相关性，并采用一致性约束推断策略以减少推断偏差和增强阶段一致性。在知名的公共数据集 Cholec80 上进行了综合验证，实验结果表明，我们的方法在定量和定性上都优于现有的最先进方法，并实现了每秒 66 帧的推断速率。