Automated diagnosis of eczema from digital camera images is crucial for
developing applications that allow patients to self-monitor their recovery. An
important component of this is the segmentation of eczema region from such
images. Current methods for eczema segmentation rely on deep neural networks
such as convolutional (CNN)-based U-Net or transformer-based Swin U-Net. While
effective, these methods require high volume of annotated data, which can be
difficult to obtain. Here, we investigate the capabilities of visual in-context
learning that can perform few-shot eczema segmentation with just a handful of
examples and without any need for retraining models. Specifically, we propose a
strategy for applying in-context learning for eczema segmentation with a
generalist vision model called SegGPT. When benchmarked on a dataset of
annotated eczema images, we show that SegGPT with just 2 representative example
images from the training dataset performs better (mIoU: 36.69) than a CNN U-Net
trained on 428 images (mIoU: 32.60). We also discover that using more number of
examples for SegGPT may in fact be harmful to its performance. Our result
highlights the importance of visual in-context learning in developing faster
and better solutions to skin imaging tasks. Our result also paves the way for
developing inclusive solutions that can cater to minorities in the demographics
who are typically heavily under-represented in the training data.

通过应用上下文学习的方法，我们提出了一种几乎不需要重新训练模型，只需要少量示例图像即可进行湿疹分割的策略，与使用 428 个图像进行训练的 CNN U-Net 相比，使用训练数据集中仅有的 2 个代表图像训练的 SegGPT 表现更好（mIoU: 36.69 vs. 32.60），并发现在 SegGPT 中使用更多的示例图像可能对其性能有害，这强调了在皮肤成像任务中发展更快更好的解决方案时，视觉上下文学习的重要性，我们的结果还为开发能够满足训练数据中通常严重不足的少数群体的包容性解决方案铺平了道路。