In machine learning, generative modeling aims to learn to generate new data
statistically similar to the training data distribution. In this paper, we
survey learning generative models under limited data, few shots and zero shot,
referred to as Generative Modeling under Data Constraint (GM-DC). This is an
important topic when data acquisition is challenging, e.g. healthcare
applications. We discuss background, challenges, and propose two taxonomies:
one on GM-DC tasks and another on GM-DC approaches. Importantly, we study
interactions between different GM-DC tasks and approaches. Furthermore, we
highlight research gaps, research trends, and potential avenues for future
exploration. Project website: this https URL

在机器学习中，生成建模旨在学习以统计上类似于训练数据分布的新数据。本文调查了在有限数据、少样本和零样本条件下学习生成模型，称为数据约束下的生成建模（GM-DC）。我们讨论了背景、挑战，并提出了两种分类方法：一种是 GM-DC 任务的分类，另一种是 GM-DC 方法的分类。重要的是，我们研究了不同 GM-DC 任务和方法之间的相互作用。此外，我们强调了研究的差距、趋势和未来探讨的潜在途径。项目网站：https://this URL

有限数据、少样本和零样本生成建模调查

A Survey on Generative Modeling with Limited Data, Few Shots, and Zero  Shot

A universal classification model aims to generalize to diverse classification
tasks in both zero and few shot settings. A promising way toward universal
classification is to cast heterogeneous data formats into a dataset-agnostic
"meta-task" (e.g., textual entailment, question answering) then pretrain a
model on the combined meta dataset. The existing work is either pretrained on
specific subsets of classification tasks, or pretrained on both classification
and generation data but the model could not fulfill its potential in
universality and reliability. These also leave a massive amount of annotated
data under-exploited. To fill these gaps, we propose ConEntail, a new framework
for universal zero and few shot classification with supervised contrastive
pretraining. Our unified meta-task for classification is based on nested
entailment. It can be interpreted as "Does sentence a entails [sentence b
entails label c]". This formulation enables us to make better use of 57
annotated classification datasets for supervised contrastive pretraining and
universal evaluation. In this way, ConEntail helps the model (1) absorb
knowledge from different datasets, and (2) gain consistent performance gain
with more pretraining data. In experiments, we compare our model with
discriminative and generative models pretrained on the same dataset. The
results confirm that our framework effectively exploits existing annotated data
and consistently outperforms baselines in both zero (9.4% average improvement)
and few shot settings (3.5% average improvement).

提出了一种基于超级对比学习预处理的新框架 ConEntail，它使用统一的元任务进行分类，该任务基于嵌套蕴含。实验结果表明，该框架可以有效地利用现有的注释数据，在零和少量样本情况下始终优于基线 (分别达到 9.4% 和 3.5% 的平均改进)。