Hematoxylin and Eosin (H&E) staining is the most commonly used for disease
diagnosis and tumor recurrence tracking. Hematoxylin excels at highlighting
nuclei, whereas eosin stains the cytoplasm. However, H&E stain lacks details
for differentiating different types of cells relevant to identifying the grade
of the disease or response to specific treatment variations. Pathologists
require special immunohistochemical (IHC) stains that highlight different cell
types. These stains help in accurately identifying different regions of disease
growth and their interactions with the cell's microenvironment. The advent of
deep learning models has made Image-to-Image (I2I) translation a key research
area, reducing the need for expensive physical staining processes. Pix2Pix and
CycleGAN are still the most commonly used methods for virtual staining
applications. However, both suffer from hallucinations or staining
irregularities when H&E stain has less discriminate information about the
underlying cells IHC needs to highlight (e.g.,CD3 lymphocytes). Diffusion
models are currently the state-of-the-art models for image generation and
conditional generation tasks. However, they require extensive and diverse
datasets (millions of samples) to converge, which is less feasible for virtual
staining applications.Inspired by the success of multitask deep learning models
for limited dataset size, we propose StainDiffuser, a novel multitask dual
diffusion architecture for virtual staining that converges under a limited
training budget. StainDiffuser trains two diffusion processes simultaneously:
(a) generation of cell-specific IHC stain from H&E and (b) H&E-based cell
segmentation using coarse segmentation only during training. Our results show
that StainDiffuser produces high-quality results for easier (CK8/18,epithelial
marker) and difficult stains(CD3, Lymphocytes).

StainDiffuser 是一种新颖的多任务双扩散架构，用于通过有限的训练预算实现虚拟染色，能同时训练细胞特异性的免疫组织化学染色和基于 HE 的细胞分割。

StainDiffuser: 虚拟染色的多任务双扩散模型

StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining

Understanding the way cells communicate, co-locate, and interrelate is
essential to understanding human physiology. Hematoxylin and eosin (H&E)
staining is ubiquitously available both for clinical studies and research. The
Colon Nucleus Identification and Classification (CoNIC) Challenge has recently
innovated on robust artificial intelligence labeling of six cell types on H&E
stains of the colon. However, this is a very small fraction of the number of
potential cell classification types. Specifically, the CoNIC Challenge is
unable to classify epithelial subtypes (progenitor, endocrine, goblet),
lymphocyte subtypes (B, helper T, cytotoxic T), or connective subtypes
(fibroblasts, stromal). In this paper, we propose to use inter-modality
learning to label previously un-labelable cell types on virtual H&E. We
leveraged multiplexed immunofluorescence (MxIF) histology imaging to identify
14 subclasses of cell types. We performed style transfer to synthesize virtual
H&E from MxIF and transferred the higher density labels from MxIF to these
virtual H&E images. We then evaluated the efficacy of learning in this
approach. We identified helper T and progenitor nuclei with positive predictive
values of $0.34 \pm 0.15$ (prevalence $0.03 \pm 0.01$) and $0.47 \pm 0.1$
(prevalence $0.07 \pm 0.02$) respectively on virtual H&E. This approach
represents a promising step towards automating annotation in digital pathology.

利用多模态学习在数字病理学中为先前无法分类的细胞类型自动标注，代表着自动标注数字病理学的有希望的进展。

核子亚型分类的跨模态学习

Nucleus subtype classification using inter-modality learning

Hematoxylin and Eosin (H&E) staining is a widely used sample preparation
procedure for enhancing the saturation of tissue sections and the contrast
between nuclei and cytoplasm in histology images for medical diagnostics.
However, various factors, such as the differences in the reagents used, result
in high variability in the colors of the stains actually recorded. This
variability poses a challenge in achieving generalization for machine-learning
based computer-aided diagnostic tools. To desensitize the learned models to
stain variations, we propose the Generative Stain Augmentation Network (G-SAN)
-- a GAN-based framework that augments a collection of cell images with
simulated yet realistic stain variations. At its core, G-SAN uses a novel and
highly computationally efficient Laplacian Pyramid (LP) based generator
architecture, that is capable of disentangling stain from cell morphology.
Through the task of patch classification and nucleus segmentation, we show that
using G-SAN-augmented training data provides on average 15.7% improvement in F1
score and 7.3% improvement in panoptic quality, respectively. Our code is
available at this https URL

提出使用生成对抗网络 (GAN) 的 Generative Stain Augmentation Network (G-SAN) 来消除组织切片染色不同的变异性对机器学习诊断工具的影响，并通过补充染色变异性数据，使得机器学习模型更加健壮。通过组织学图像的核分类和分割任务的结果，证明使用 G-SAN 的增强数据平均提高了 F1 得分 15.7％和齐柏林物质的质量 7.3％。