2D/3D image registration to align a 3D volume and 2D X-ray images is a
challenging problem due to its ill-posed nature and various artifacts presented
in 2D X-ray images. In this paper, we propose a multi-agent system with an auto
attention mechanism for robust and efficient 2D/3D image registration.
Specifically, an individual agent is trained with dilated Fully Convolutional
Network (FCN) to perform registration in a Markov Decision Process (MDP) by
observing a local region, and the final action is then taken based on the
proposals from multiple agents and weighted by their corresponding confidence
levels. The contributions of this paper are threefold. First, we formulate
2D/3D registration as a MDP with observations, actions, and rewards properly
defined with respect to X-ray imaging systems. Second, to handle various
artifacts in 2D X-ray images, multiple local agents are employed efficiently
via FCN-based structures, and an auto attention mechanism is proposed to favor
the proposals from regions with more reliable visual cues. Third, a dilated
FCN-based training mechanism is proposed to significantly reduce the Degree of
Freedom in the simulation of registration environment, and drastically improve
training efficiency by an order of magnitude compared to standard CNN-based
training method. We demonstrate that the proposed method achieves high
robustness on both spine cone beam Computed Tomography data with a low
signal-to-noise ratio and data from minimally invasive spine surgery where
severe image artifacts and occlusions are presented due to metal screws and
guide wires, outperforming other state-of-the-art methods (single agent-based
and optimization-based) by a large margin.

该论文提出了一种多智能体系统和自注意力机制相结合的方法，将 2D / 3D 图像配准作为马尔可夫决策过程来解决，通过多个基于全卷积网络的智能体的提案及对应置信度的加权来取得了超过基于单智能体和基于优化方法的最新技术的优异成果。