The distribution of subpopulations is an important property hidden within a
dataset. Uncovering and analyzing the subpopulation distribution within
datasets provides a comprehensive understanding of the datasets, standing as a
powerful tool beneficial to various downstream tasks, including Dataset
Subpopulation Organization, Subpopulation Shift, and Slice Discovery. Despite
its importance, there has been no work that systematically explores the
subpopulation distribution of datasets to our knowledge. To address the
limitation and solve all the mentioned tasks in a unified way, we introduce a
novel concept of subpopulation structures to represent, analyze, and utilize
subpopulation distributions within datasets. To characterize the structures in
an interpretable manner, we propose the Subpopulation Structure Discovery with
Large Language Models (SSD-LLM) framework, which employs world knowledge and
instruction-following capabilities of Large Language Models (LLMs) to
linguistically analyze informative image captions and summarize the structures.
Furthermore, we propose complete workflows to address downstream tasks, named
Task-specific Tuning, showcasing the application of the discovered structure to
a spectrum of subpopulation-related tasks, including dataset subpopulation
organization, subpopulation shift, and slice discovery. Furthermore, we propose
complete workflows to address downstream tasks, named Task-specific Tuning,
showcasing the application of the discovered structure to a spectrum of
subpopulation-related tasks, including dataset subpopulation organization,
subpopulation shift, and slice discovery.

探索数据集的子群体分布并应用于各种下游任务，介绍了子群体结构的概念及其在大型语言模型中的分析框架以及应用于数据集子群体组织、子群体迁移和切片发现等任务的完整工作流程。