Nuclei detection and segmentation in hematoxylin and eosin-stained (H&E)
tissue images are important clinical tasks and crucial for a wide range of
applications. However, it is a challenging task due to nuclei variances in
staining and size, overlapping boundaries, and nuclei clustering. While
convolutional neural networks have been extensively used for this task, we
explore the potential of Transformer-based networks in this domain. Therefore,
we introduce a new method for automated instance segmentation of cell nuclei in
digitized tissue samples using a deep learning architecture based on Vision
Transformer called CellViT. CellViT is trained and evaluated on the PanNuke
dataset, which is one of the most challenging nuclei instance segmentation
datasets, consisting of nearly 200,000 annotated Nuclei into 5 clinically
important classes in 19 tissue types. We demonstrate the superiority of
large-scale in-domain and out-of-domain pre-trained Vision Transformers by
leveraging the recently published Segment Anything Model and a ViT-encoder
pre-trained on 104 million histological image patches - achieving
state-of-the-art nuclei detection and instance segmentation performance on the
PanNuke dataset with a mean panoptic quality of 0.51 and an F1-detection score
of 0.83. The code is publicly available at this https URL

介绍了一种基于 Vision Transformer 的新方法 CellViT，利用大规模的预训练模型，在 PanNuke 数据集上实现对细胞核的自动检测和实例分割的最新性能，并获得了 0.51 的平均全景质量和 0.83 的 F1 检测分数。