Purpose: Real-time surgical tool tracking is a core component of the future
intelligent operating room (OR), because it is highly instrumental to analyze
and understand the surgical activities. Current methods for surgical tool
tracking in videos need to be trained on data in which the spatial positions of
the tools are manually annotated. Generating such training data is difficult
and time-consuming. Instead, we propose to use solely binary presence
annotations to train a tool tracker for laparoscopic videos. Methods: The
proposed approach is composed of a CNN + Convolutional LSTM (ConvLSTM) neural
network trained end-to-end, but weakly supervised on tool binary presence
labels only. We use the ConvLSTM to model the temporal dependencies in the
motion of the surgical tools and leverage its spatio-temporal ability to smooth
the class peak activations in the localization heat maps (Lh-maps).
Results: We build a baseline tracker on top of the CNN model and demonstrate
that our approach based on the ConvLSTM outperforms the baseline in tool
presence detection, spatial localization, and motion tracking by over 5.0%,
13.9%, and 12.6%, respectively.
Conclusions: In this paper, we demonstrate that binary presence labels are
sufficient for training a deep learning tracking model using our proposed
method. We also show that the ConvLSTM can leverage the spatio-temporal
coherence of consecutive image frames across a surgical video to improve tool
presence detection, spatial localization, and motion tracking.
keywords: Surgical workflow analysis, tool tracking, weak supervision,
spatio-temporal coherence, ConvLSTM, endoscopic videos

本文介绍了一种仅使用二分类标记来训练内窥镜视频工具跟踪器的方法，这种方法利用了连续图像帧的时空一致性，基于 ConvLSTM 对工具的存在检测、空间定位和运动跟踪进行了改进。