Intensive Care in-hospital mortality prediction has various clinical applications. Neural prediction models, especially when capitalising on clinical notes, have been put forward as improvement on currently existing models. However, to be acceptable these models should be performant and transparent. This work studies different attention mechanisms for clinical neural prediction models in terms of their discrimination and calibration. Specifically, we investigate sparse attention as an alternative to dense attention weights in the task of in-hospital mortality prediction from clinical notes. We evaluate the attention mechanisms based on: i) local self-attention over words in a sentence, and ii) global self-attention with a transformer architecture across sentences. We demonstrate that the sparse mechanism approach outperforms the dense one for the local self-attention in terms of predictive performance with a publicly available dataset, and puts higher attention to prespecified relevant directive words. The performance at the sentence level, however, deteriorates as sentences including the influential directive words tend to be dropped all together.

本文研究了临床神经预测模型中的不同注意力机制在区分度和校准性方面的表现，特别是将稀疏注意力作为在临床记录中预测住院病死率的替代方法，在公开数据集上表现出了优异的预测性能，并且在确定的相关指令词上更加注意力集中。然而，在句子选择上可能会出现问题。

使用字级别和句级别稀疏注意力的临床笔记死亡率预测模型