In certain types of cancerous tissue, mitotic count has been shown to be
associated with tumor proliferation, poor prognosis, and therapeutic
resistance. Due to the high inter-rater variability of mitotic counting by
pathologists, convolutional neural networks (CNNs) have been employed to reduce
the subjectivity of mitosis detection in hematoxylin and eosin (H&E)-stained
whole slide images. However, most existing models have performance that lags
behind expert panel review and only incorporate visual information. In this
work, we demonstrate that pre-trained large-scale vision-language models that
leverage both visual features and natural language improve mitosis detection
accuracy. We formulate the mitosis detection task as an image captioning task
and a visual question answering (VQA) task by including metadata such as tumor
and scanner types as context. The effectiveness of our pipeline is demonstrated
via comparison with various baseline models using 9,501 mitotic figures and
11,051 hard negatives (non-mitotic figures that are difficult to characterize)
from the publicly available Mitosis Domain Generalization Challenge (MIDOG22)
dataset.

癌症组织中，有丝分裂计数与肿瘤增殖、预后不良和药物抵抗有关。我们采用卷积神经网络（CNN）来降低病理学家之间有丝分裂计数的主观性，并结合视觉特征和自然语言，将有丝分裂检测任务转化为图像描述任务和视觉问答任务，以提高检测准确性。通过与基线模型在 9,501 个有丝分裂图像和 11,051 个难以表征的负例图像（非有丝分裂图像）上的比较，证明了我们方法的有效性。