Prosthetic Joint Infection (PJI) is a prevalent and severe complication
characterized by high diagnostic challenges. Currently, a unified diagnostic
standard incorporating both computed tomography (CT) images and numerical text
data for PJI remains unestablished, owing to the substantial noise in CT images
and the disparity in data volume between CT images and text data. This study
introduces a diagnostic method, HGT, based on deep learning and multimodal
techniques. It effectively merges features from CT scan images and patients'
numerical text data via a Unidirectional Selective Attention (USA) mechanism
and a graph convolutional network (GCN)-based feature fusion network. We
evaluated the proposed method on a custom-built multimodal PJI dataset,
assessing its performance through ablation experiments and interpretability
evaluations. Our method achieved an accuracy (ACC) of 91.4\% and an area under
the curve (AUC) of 95.9\%, outperforming recent multimodal approaches by 2.9\%
in ACC and 2.2\% in AUC, with a parameter count of only 68M. Notably, the
interpretability results highlighted our model's strong focus and localization
capabilities at lesion sites. This proposed method could provide clinicians
with additional diagnostic tools to enhance accuracy and efficiency in clinical
practice.

本研究提出了一种基于深度学习和多模态技术的诊断方法 HGT，通过单向选择性注意机制和基于图卷积网络（GCN）的特征融合网络，有效地将 CT 扫描图像和患者的数字文本数据特征融合。经过消融实验和可解释性评估，该方法在自定义的多模态 PJI 数据集上取得了 91.4％ 的准确率和 95.9％ 的曲线下面积（AUC），在 ACC 和 AUC 上分别优于最近的多模态方法，并仅使用了 68M 的参数计数。该方法可以为临床医生提供额外的诊断工具，以提高诊断精度和效率。