Despite their black-box nature, deep learning models are extensively used in
image-based drug discovery to extract feature vectors from single cells in
microscopy images. To better understand how these networks perform
representation learning, we employ visual explainability techniques (e.g.,
Grad-CAM). Our analyses reveal several mechanisms by which supervised models
cheat, exploiting biologically irrelevant pixels when extracting morphological
features from images, such as noise in the background. This raises doubts
regarding the fidelity of learned single-cell representations and their
relevance when investigating downstream biological questions. To address this
misalignment between researcher expectations and machine behavior, we introduce
Grad-CAMO, a novel single-cell interpretability score for supervised feature
extractors. Grad-CAMO measures the proportion of a model's attention that is
concentrated on the cell of interest versus the background. This metric can be
assessed per-cell or averaged across a validation set, offering a tool to audit
individual features vectors or guide the improved design of deep learning
architectures. Importantly, Grad-CAMO seamlessly integrates into existing
workflows, requiring no dataset or model modifications, and is compatible with
both 2D and 3D Cell Painting data. Additional results are available at
this https URL

深度学习模型在基于图像的药物发现中被广泛用于从显微镜图像中提取单个细胞的特征向量。通过使用可视解释性技术（例如 Grad-CAM），我们发现了监督模型在提取形态特征时作弊的几种机制，即从图像中提取生物学无关的像素，如背景噪音。为了解决研究者期望和机器行为之间的错位，我们引入了 Grad-CAMO，一种用于监督特征提取器的新型单细胞可解释性评分。Grad-CAMO 衡量模型关注细胞本体与背景的比例，可对每个细胞或验证集进行评估，从而提供对个体特征向量的审计或指导深度学习架构的改进设计工具。重要的是，Grad-CAMO 可以无缝集成到现有的工作流程中，无需对数据集或模型进行修改，且与 2D 和 3D 细胞图像数据兼容。