Understanding the relationship between figures and text is key to scientific
document understanding. Medical figures in particular are quite complex, often
consisting of several subfigures (75% of figures in our dataset), with detailed
text describing their content. Previous work studying figures in scientific
papers focused on classifying figure content rather than understanding how
images relate to the text. To address challenges in figure retrieval and
figure-to-text alignment, we introduce MedICaT, a dataset of medical images in
context. MedICaT consists of 217K images from 131K open access biomedical
papers, and includes captions, inline references for 74% of figures, and
manually annotated subfigures and subcaptions for a subset of figures. Using
MedICaT, we introduce the task of subfigure to subcaption alignment in compound
figures and demonstrate the utility of inline references in image-text
matching. Our data and code can be accessed at
this https URL

介绍了用于 MedICaT 医学图像的上下文数据集，其中包括来自 131k 篇开放获取生物医学论文的 217k 个图像。该数据集用于研究复合图中的子图对子标题的自动对齐，并展示了内联引用在图像文本匹配中的效用。