We propose DiRL, a Diversity-inducing Representation Learning technique for
histopathology imaging. Self-supervised learning techniques, such as
contrastive and non-contrastive approaches, have been shown to learn rich and
effective representations of digitized tissue samples with limited pathologist
supervision. Our analysis of vanilla SSL-pretrained models' attention
distribution reveals an insightful observation: sparsity in attention, i.e,
models tends to localize most of their attention to some prominent patterns in
the image. Although attention sparsity can be beneficial in natural images due
to these prominent patterns being the object of interest itself, this can be
sub-optimal in digital pathology; this is because, unlike natural images,
digital pathology scans are not object-centric, but rather a complex phenotype
of various spatially intermixed biological components. Inadequate
diversification of attention in these complex images could result in crucial
information loss. To address this, we leverage cell segmentation to densely
extract multiple histopathology-specific representations, and then propose a
prior-guided dense pretext task for SSL, designed to match the multiple
corresponding representations between the views. Through this, the model learns
to attend to various components more closely and evenly, thus inducing adequate
diversification in attention for capturing context rich representations.
Through quantitative and qualitative analysis on multiple tasks across cancer
types, we demonstrate the efficacy of our method and observe that the attention
is more globally distributed.

我们提出了 DiRL，一种多样性诱导的组织病理学成像表示学习技术。通过细胞分割密集提取多个组织病理学特定表示，并提出了一种先验引导的密集预训练任务，以匹配视图之间的多个对应表示。通过定量和定性分析肿瘤类型任务的多个任务，我们证明了我们方法的有效性，并观察到注意力更加全局分布。

注意力去稀疏化的重要性：在数字病理学表示学习中诱导多样性

Attention De-sparsification Matters: Inducing Diversity in Digital  Pathology Representation Learning

Various Seq2Seq learning models designed for machine translation were applied
for abstractive summarization task recently. Despite these models provide high
ROUGE scores, they are limited to generate comprehensive summaries with a high
level of abstraction due to its degenerated attention distribution. We
introduce Diverse Convolutional Seq2Seq Model(DivCNN Seq2Seq) using
Determinantal Point Processes methods(Micro DPPs and Macro DPPs) to produce
attention distribution considering both quality and diversity. Without breaking
the end to end architecture, DivCNN Seq2Seq achieves a higher level of
comprehensiveness compared to vanilla models and strong baselines. All the
reproducible codes and datasets are available online.

本文通过引入一种称为微观 DPPs 和宏观 DPPs 的方法，提出了一个名为 DivCNN Seq2Seq 的 Diverse Convolutional Seq2Seq 模型，用于生成注重质量和多样性的 attention distribution，从而相比传统模型和强基线模型实现更高的综合性摘要生成。

综合的和抽象的摘要：基于行列式点过程的多样化注意力

In Conclusion Not Repetition: Comprehensive Abstractive Summarization  With Diversified Attention Based On Determinantal Point Processes

Squeeze-and-Excitation (SE) block presents a channel attention mechanism for
modeling global context via explicitly capturing dependencies across channels.
However, we are still far from understanding how the SE block works. In this
work, we first revisit the SE block, and then present a detailed empirical
study of the relationship between global context and attention distribution,
based on which we propose a simple yet effective module, called Linear Context
Transform (LCT) block. We divide all channels into different groups and
normalize the globally aggregated context features within each channel group,
reducing the disturbance from irrelevant channels. Through linear transform of
the normalized context features, we model global context for each channel
independently. The LCT block is extremely lightweight and easy to be plugged
into different backbone models while with negligible parameters and
computational burden increase. Extensive experiments show that the LCT block
outperforms the SE block in image classification task on the ImageNet and
object detection/segmentation on the COCO dataset with different backbone
models. Moreover, LCT yields consistent performance gains over existing
state-of-the-art detection architectures, e.g., 1.5$\sim$1.7% AP$^{bbox}$ and
1.0$\sim$1.2% AP$^{mask}$ improvements on the COCO benchmark, irrespective of
different baseline models of varied capacities. We hope our simple yet
effective approach will shed some light on future research of attention-based
models.

本文提出 Linear Context Transform (LCT) 块，一种基于通道分组和线性变换的轻量级模块，可用于不同主干模型中。与 Squeeze-and-Excitation 块相比，LCT 块在 ImageNet 图像分类和 COCO 数据集目标检测 / 分割中，均获得了更好的性能，且对于不同基线模型均具有较高的一致性表现。

线性上下文转换块

Linear Context Transform Block

Explaining and interpreting the decisions of recommender systems are becoming
extremely relevant both, for improving predictive performance, and providing
valid explanations to users. While most of the recent interest has focused on
providing local explanations, there has been a much lower emphasis on studying
the effects of model dynamics and its impact on explanation. In this paper, we
perform a focused study on the impact of model interpretability in the context
of calibration. Specifically, we address the challenges of both over-confident
and under-confident predictions with interpretability using attention
distribution. Our results indicate that the means of using attention
distributions for interpretability are highly unstable for un-calibrated
models. Our empirical analysis on the stability of attention distribution
raises questions on the utility of attention for explainability.

本文研究了在校准背景下，使用关注力分布实现模型解释性的难点，发现没有校准的模型使用关注力分布的方式实现解释性是不稳定的，这引发了关于关注力在可解释性中的实用性的疑问。