As deep learning has become the state-of-the-art for computer-assisted
diagnosis, interpretability of the automatic decisions is crucial for clinical
deployment. While various methods were proposed in this domain, visual
attention maps of clinicians during radiological screening offer a unique asset
to provide important insights and can potentially enhance the quality of
computer-assisted diagnosis. With this paper, we introduce a novel
deep-learning framework for joint disease diagnosis and prediction of
corresponding visual saliency maps for chest X-ray scans. Specifically, we
designed a novel dual-encoder multi-task UNet, which leverages both a
DenseNet201 backbone and a Residual and Squeeze-and-Excitation block-based
encoder to extract diverse features for saliency map prediction, and a
multi-scale feature-fusion classifier to perform disease classification. To
tackle the issue of asynchronous training schedules of individual tasks in
multi-task learning, we proposed a multi-stage cooperative learning strategy,
with contrastive learning for feature encoder pretraining to boost performance.
Experiments show that our proposed method outperformed existing techniques for
chest X-ray diagnosis and the quality of visual saliency map prediction.

使用新型深度学习框架，结合医生的视觉注意力图，在胸部 X 射线扫描中实现关节疾病的诊断和预测对应的视觉显著性图，提高计算机辅助诊断质量。

多阶段合作学习的胸部 X 光透视与临床视觉注意预测：增强解释性

Joint chest X-ray diagnosis and clinical visual attention prediction  with multi-stage cooperative learning: enhancing interpretability

We consider the problem of composed image retrieval that takes an input query
consisting of an image and a modification text indicating the desired changes
to be made on the image and retrieves images that match these changes. Current
state-of-the-art techniques that address this problem use global features for
the retrieval, resulting in incorrect localization of the regions of interest
to be modified because of the global nature of the features, more so in cases
of real-world, in-the-wild images. Since modifier texts usually correspond to
specific local changes in an image, it is critical that models learn local
features to be able to both localize and retrieve better. To this end, our key
novelty is a new gradient-attention-based learning objective that explicitly
forces the model to focus on the local regions of interest being modified in
each retrieval step. We achieve this by first proposing a new visual image
attention computation technique, which we call multi-modal gradient attention
(MMGrad) that is explicitly conditioned on the modifier text. We next
demonstrate how MMGrad can be incorporated into an end-to-end model training
strategy with a new learning objective that explicitly forces these MMGrad
attention maps to highlight the correct local regions corresponding to the
modifier text. By training retrieval models with this new loss function, we
show improved grounding by means of better visual attention maps, leading to
better explainability of the models as well as competitive quantitative
retrieval performance on standard benchmark datasets.

我们提出了一种基于梯度注意的学习目标，用于解决组合图像检索问题。通过引入新的注意力计算技术和学习目标，我们展示了对图像区域进行更准确的定位和改进，并通过标准基准数据集显示了更好的可解释性和竞争性的定量检索性能。

多模态梯度注意力学习用于可解释的组合图像检索

Learning with Multi-modal Gradient Attention for Explainable Composed  Image Retrieval

Understanding and explaining deep learning models is an imperative task.
Towards this, we propose a method that obtains gradient-based certainty
estimates that also provide visual attention maps. Particularly, we solve for
visual question answering task. We incorporate modern probabilistic deep
learning methods that we further improve by using the gradients for these
estimates. These have two-fold benefits: a) improvement in obtaining the
certainty estimates that correlate better with misclassified samples and b)
improved attention maps that provide state-of-the-art results in terms of
correlation with human attention regions. The improved attention maps result in
consistent improvement for various methods for visual question answering.
Therefore, the proposed technique can be thought of as a recipe for obtaining
improved certainty estimates and explanations for deep learning models. We
provide detailed empirical analysis for the visual question answering task on
all standard benchmarks and comparison with state of the art methods.

这篇研究提出了一种方法，通过基于梯度的确定性估计来解决视觉问答任务，进一步利用这些估计来获得可视化的注意力图，从而提高深度学习模型的确定性估计和解释能力，并在各项标准基准测试中提供实证分析及与现有方法的比较