Attention mechanism is effective in both focusing the deep learning models on relevant features and interpreting them. However, attentions may be unreliable since the networks that generate them are often trained in a weakly-supervised manner. To overcome this limitation, we introduce the notion of input-dependent uncertainty to the attention mechanism, such that it generates attention for each feature with varying degrees of noise based on the given input, to learn larger variance on instances it is uncertain about. We learn this Uncertainty-aware Attention (UA) mechanism using variational inference, and validate it on various risk prediction tasks from electronic health records on which our model significantly outperforms existing attention models. The analysis of the learned attentions shows that our model generates attentions that comply with clinicians' interpretation, and provide richer interpretation via learned variance. Further evaluation of both the accuracy of the uncertainty calibration and the prediction performance with "I don't know" decision show that UA yields networks with high reliability as well.

通过引入输入相关的不确定度来学习输入不确定的实例的更大方差， 我们使用变分推断学习了不确定性感知注意力机制， 并在不同的高风险预测任务中验证了其有效性。进一步的评估表明，我们的模型生成符合临床医生解释的注意力，并通过学习方差提供更丰富的解释。

基於不確定性的注意力機制用於可靠的解釋和預測