Assessing tumor response to systemic therapies is one of the main
applications of PET/CT. Routinely, only a small subset of index lesions out of
multiple lesions is analyzed. However, this operator dependent selection may
bias the results due to possible significant inter-metastatic heterogeneity of
response to therapy. Automated, AI based approaches for lesion tracking hold
promise in enabling the analysis of many more lesions and thus providing a
better assessment of tumor response. This work introduces a Siamese CNN
approach for lesion tracking between PET/CT scans. Our approach is applied on
the laborious task of tracking a high number of bone lesions in full-body
baseline and follow-up [68Ga]Ga- or [18F]F-PSMA PET/CT scans after two cycles
of [177Lu]Lu-PSMA therapy of metastatic castration resistant prostate cancer
patients. Data preparation includes lesion segmentation and affine
registration. Our algorithm extracts suitable lesion patches and forwards them
into a Siamese CNN trained to classify the lesion patch pairs as corresponding
or non-corresponding lesions. Experiments have been performed with different
input patch types and a Siamese network in 2D and 3D. The CNN model
successfully learned to classify lesion assignments, reaching a lesion tracking
accuracy of 83 % in its best configuration with an AUC = 0.91. For remaining
lesions the pipeline accomplished a re-identification rate of 89 %. We proved
that a CNN may facilitate the tracking of multiple lesions in PSMA PET/CT
scans. Future clinical studies are necessary if this improves the prediction of
the outcome of therapies.

通过 Siamese CNN 方法，应用于 PSMA PET/CT 扫描，成功实现了多发性病灶的追踪，该方法在预测治疗结果方面表现出潜力。

PET/CT 图像中的 AI 病变跟踪：基于 Siamese 的 CNN 管道应用于 PSMA PET/CT 扫描

Towards AI Lesion Tracking in PET/CT Imaging: A Siamese-based CNN  Pipeline applied on PSMA PET/CT Scans

Monitoring treatment response in longitudinal studies plays an important role
in clinical practice. Accurately identifying lesions across serial imaging
follow-up is the core to the monitoring procedure. Typically this incorporates
both image and anatomical considerations. However, matching lesions manually is
labor-intensive and time-consuming. In this work, we present deep lesion
tracker (DLT), a deep learning approach that uses both appearance- and
anatomical-based signals. To incorporate anatomical constraints, we propose an
anatomical signal encoder, which prevents lesions being matched with visually
similar but spurious regions. In addition, we present a new formulation for
Siamese networks that avoids the heavy computational loads of 3D
cross-correlation. To present our network with greater varieties of images, we
also propose a self-supervised learning (SSL) strategy to train trackers with
unpaired images, overcoming barriers to data collection. To train and evaluate
our tracker, we introduce and release the first lesion tracking benchmark,
consisting of 3891 lesion pairs from the public DeepLesion database. The
proposed method, DLT, locates lesion centers with a mean error distance of 7
mm. This is 5% better than a leading registration algorithm while running 14
times faster on whole CT volumes. We demonstrate even greater improvements over
detector or similarity-learning alternatives. DLT also generalizes well on an
external clinical test set of 100 longitudinal studies, achieving 88% accuracy.
Finally, we plug DLT into an automatic tumor monitoring workflow where it leads
to an accuracy of 85% in assessing lesion treatment responses, which is only
0.46% lower than the accuracy of manual inputs.

本研究提出了基于深度学习的深度损伤跟踪器（DLT），该方法解决了长期研究中监测治疗反应的核心问题，即如何准确地对大量数据进行图像识别和解剖学研究，同时提高检测的速度和精度，该方法已在公共 DeepLesion 数据库上进行了测试和验证，并成功应用于肿瘤监测工作流程中。