A major barrier to deploying healthcare AI models is their trustworthiness.
One form of trustworthiness is a model's robustness across different subgroups:
while existing models may exhibit expert-level performance on aggregate
metrics, they often rely on non-causal features, leading to errors in hidden
subgroups. To take a step closer towards trustworthy seizure onset detection
from EEG, we propose to leverage annotations that are produced by healthcare
personnel in routine clinical workflows -- which we refer to as workflow notes
-- that include multiple event descriptions beyond seizures. Using workflow
notes, we first show that by scaling training data to an unprecedented level of
68,920 EEG hours, seizure onset detection performance significantly improves
(+12.3 AUROC points) compared to relying on smaller training sets with
expensive manual gold-standard labels. Second, we reveal that our binary
seizure onset detection model underperforms on clinically relevant subgroups
(e.g., up to a margin of 6.5 AUROC points between pediatrics and adults), while
having significantly higher false positives on EEG clips showing
non-epileptiform abnormalities compared to any EEG clip (+19 FPR points). To
improve model robustness to hidden subgroups, we train a multilabel model that
classifies 26 attributes other than seizures, such as spikes, slowing, and
movement artifacts. We find that our multilabel model significantly improves
overall seizure onset detection performance (+5.9 AUROC points) while greatly
improving performance among subgroups (up to +8.3 AUROC points), and decreases
false positives on non-epileptiform abnormalities by 8 FPR points. Finally, we
propose a clinical utility metric based on false positives per 24 EEG hours and
find that our multilabel model improves this clinical utility metric by a
factor of 2x across different clinical settings.

为解决医疗 AI 模型可信度低的问题，该研究利用医疗人员定期工作流中的注释（称为工作流注释），提出了一种基于多标签模型的癫痫发作检测方法，通过对 26 种属性的分类，显著提高了癫痫发作检测性能并减少了误报。