In medicine, both ethical and monetary costs of incorrect predictions can be
significant, and the complexity of the problems often necessitates increasingly
complex models. Recent work has shown that changing just the random seed is
enough for otherwise well-tuned deep neural networks to vary in their
individual predicted probabilities. In light of this, we investigate the role
of model uncertainty methods in the medical domain. Using RNN ensembles and
various Bayesian RNNs, we show that population-level metrics, such as AUC-PR,
AUC-ROC, log-likelihood, and calibration error, do not capture model
uncertainty. Meanwhile, the presence of significant variability in
patient-specific predictions and optimal decisions motivates the need for
capturing model uncertainty. Understanding the uncertainty for individual
patients is an area with clear clinical impact, such as determining when a
model decision is likely to be brittle. We further show that RNNs with only
Bayesian embeddings can be a more efficient way to capture model uncertainty
compared to ensembles, and we analyze how model uncertainty is impacted across
individual input features and patient subgroups.

研究在医学上，使用其他深度神经网络模型方法来捕捉模型的不确定性，以确定模型决策是否在个体患者上具有一定程度的可靠性，同样通过贝叶斯嵌入，RNN 模型可以更有效地捕获模型的不确定性，研究分析模型不确定性受各种输入特征以及患者亚组的影响。