Surgery to treat elderly hip fracture patients may cause complications that
can lead to early mortality. An early warning system for complications could
provoke clinicians to monitor high-risk patients more carefully and address
potential complications early, or inform the patient. In this work, we develop
a multimodal deep-learning model for post-operative mortality prediction using
pre-operative and per-operative data from elderly hip fracture patients.
Specifically, we include static patient data, hip and chest images before
surgery in pre-operative data, vital signals, and medications administered
during surgery in per-operative data. We extract features from image modalities
using ResNet and from vital signals using LSTM. Explainable model outcomes are
essential for clinical applicability, therefore we compute Shapley values to
explain the predictions of our multimodal black box model. We find that i)
Shapley values can be used to estimate the relative contribution of each
modality both locally and globally, and ii) a modified version of the chain
rule can be used to propagate Shapley values through a sequence of models
supporting interpretable local explanations. Our findings imply that a
multimodal combination of black box models can be explained by propagating
Shapley values through the model sequence.

通过使用预手术和术中数据，我们开发了一个多模态深度学习模型，用于预测老年髋关节骨折患者术后死亡率，并通过计算 Shapley 值解释了我们的模型预测结果。