Recognizing various surgical tools, actions and phases from surgery videos is
an important problem in computer vision with exciting clinical applications.
Existing deep-learning-based methods for this problem either process each
surgical video as a series of independent images without considering their
dependence, or rely on complicated deep learning models to count for dependence
of video frames. In this study, we revealed from exploratory data analysis that
surgical videos enjoy relatively simple semantic structure, where the presence
of surgical phases and tools can be well modeled by a compact hidden Markov
model (HMM). Based on this observation, we propose an HMM-stabilized deep
learning method for tool presence detection. A wide range of experiments
confirm that the proposed approaches achieve better performance with lower
training and running costs, and support more flexible ways to construct and
utilize training data in scenarios where not all surgery videos of interest are
extensively labelled. These results suggest that popular deep learning
approaches with over-complicated model structures may suffer from inefficient
utilization of data, and integrating ingredients of deep learning and
statistical learning wisely may lead to more powerful algorithms that enjoy
competitive performance, transparent interpretation and convenient model
training simultaneously.

经过探索性数据分析，我们发现手术视频具有相对简单的语义结构，我们提出了一种基于隐马尔可夫模型（HMM）的深度学习方法，用于工具存在性检测，结果表明，与复杂模型结构的流行深度学习方法相比，我们的方法在性能、训练和运行成本以及数据利用方面都更优，智能地将深度学习和统计学相结合可能会导致更强大的算法，同时享有竞争性能、透明解释和便捷的模型训练方式。