We developed a deep learning classifier of rectal cancer response (tumor vs.
no-tumor) to total neoadjuvant treatment (TNT) from endoscopic images acquired
before, during, and following TNT. We further evaluated the network's ability
in a near out-of-distribution (OOD) problem to identify local regrowth (LR)
from follow-up endoscopy images acquired several months to years after
completing TNT. We addressed endoscopic image variability by using optimal mass
transport-based image harmonization. We evaluated multiple training
regularization schemes to study the ResNet-50 network's in-distribution and
near-OOD generalization ability. Test time augmentation resulted in the most
considerable accuracy improvement. Image harmonization resulted in slight
accuracy improvement for the near-OOD cases. Our results suggest that
off-the-shelf deep learning classifiers can detect rectal cancer from
endoscopic images at various stages of therapy for surveillance.

我们开发了一个深度学习分类器，用于从内窥镜图像中识别在全新辅助治疗过程中的直肠癌反应（肿瘤与非肿瘤）。我们通过使用基于最优质量运输的图像协调方法来解决内窥镜图像的多样性，并通过评估多种训练规范化方案来研究 ResNet-50 网络在分布内和接近分布外的泛化能力。我们的结果表明，现成的深度学习分类器能够在不同阶段的疗法监测中从内窥镜图像中检测直肠癌。