A clinical artificial intelligence (AI) system is often validated on a
held-out set of data which it has not been exposed to before (e.g., data from a
different hospital with a distinct electronic health record system). This
evaluation process is meant to mimic the deployment of an AI system on data in
the wild; those which are currently unseen by the system yet are expected to be
encountered in a clinical setting. However, when data in the wild differ from
the held-out set of data, a phenomenon referred to as distribution shift, and
lack ground-truth annotations, it becomes unclear the extent to which AI-based
findings can be trusted on data in the wild. Here, we introduce SUDO, a
framework for evaluating AI systems without ground-truth annotations. SUDO
assigns temporary labels to data points in the wild and directly uses them to
train distinct models, with the highest performing model indicative of the most
likely label. Through experiments with AI systems developed for dermatology
images, histopathology patches, and clinical reports, we show that SUDO can be
a reliable proxy for model performance and thus identify unreliable
predictions. We also demonstrate that SUDO informs the selection of models and
allows for the previously out-of-reach assessment of algorithmic bias for data
in the wild without ground-truth annotations. The ability to triage unreliable
predictions for further inspection and assess the algorithmic bias of AI
systems can improve the integrity of research findings and contribute to the
deployment of ethical AI systems in medicine.

通过 SUDO 框架，可以评估无标注的数据并训练模型，识别不可靠的预测结果和评估医学中 AI 系统的算法偏差。

SUDO: 无需真实标注的评估临床人工智能系统的框架

SUDO: a framework for evaluating clinical artificial intelligence  systems without ground-truth annotations

In clinical artificial intelligence (AI), graph representation learning,
mainly through graph neural networks (GNNs), stands out for its capability to
capture intricate relationships within structured clinical datasets. With
diverse data -- from patient records to imaging -- GNNs process data
holistically by viewing modalities as nodes interconnected by their
relationships. Graph AI facilitates model transfer across clinical tasks,
enabling models to generalize across patient populations without additional
parameters or minimal re-training. However, the importance of human-centered
design and model interpretability in clinical decision-making cannot be
overstated. Since graph AI models capture information through localized neural
transformations defined on graph relationships, they offer both an opportunity
and a challenge in elucidating model rationale. Knowledge graphs can enhance
interpretability by aligning model-driven insights with medical knowledge.
Emerging graph models integrate diverse data modalities through pre-training,
facilitate interactive feedback loops, and foster human-AI collaboration,
paving the way to clinically meaningful predictions.

临床人工智能中的图表示学习，特别是通过图神经网络 (GNN)，以其捕捉结构化临床数据中复杂关系的能力而脱颖而出。图人工智能通过将模态作为通过它们的关系相互连接的节点整体地处理数据，从而促进了模型在临床任务之间的转移，使模型能够在没有额外参数或最小重新训练的情况下推广到患者群体。然而，人类中心设计和模型解释能力在临床决策中的重要性不容忽视。由于图人工智能模型通过定义在图关系上的局部神经变换捕捉信息，它们在阐明模型的合理性方面既带来机遇又带来挑战。知识图谱可以通过将模型驱动的见解与医学知识相结合来增强解释性。新兴的图模型通过预训练整合多种数据模态，促进互动反馈循环，促进人工智能和人类的合作，为临床意义的预测铺平了道路。