Medical data poses a daunting challenge for AI algorithms: it exists in many
different modalities, experiences frequent distribution shifts, and suffers
from a scarcity of examples and labels. Recent advances, including transformers
and self-supervised learning, promise a more universal approach that can be
applied flexibly across these diverse conditions. To measure and drive progress
in this direction, we present BenchMD: a benchmark that tests how
modality-agnostic methods, including architectures and training techniques
(e.g. self-supervised learning, ImageNet pretraining), perform on a diverse
array of clinically-relevant medical tasks. BenchMD combines 19 publicly
available datasets for 7 medical modalities, including 1D sensor data, 2D
images, and 3D volumetric scans. Our benchmark reflects real-world data
constraints by evaluating methods across a range of dataset sizes, including
challenging few-shot settings that incentivize the use of pretraining. Finally,
we evaluate performance on out-of-distribution data collected at different
hospitals than the training data, representing naturally-occurring distribution
shifts that frequently degrade the performance of medical AI models. Our
baseline results demonstrate that no modality-agnostic technique achieves
strong performance across all modalities, leaving ample room for improvement on
the benchmark. Code is released at this https URL .

提出了一个名为 BenchMD 的基准测试平台，旨在评估面向临床任务的多个医疗模态数据上的通用 AI 算法如何处理，包括使用 modality-agnostic 方法和 self-supervised learning 技术，研究结果表明当前没有一种方法可以在所有模态上实现出色性能。