Data generation-based zero-shot learning, although effective in training
Small Task-specific Models (STMs) via synthetic datasets generated by
Pre-trained Language Models (PLMs), is often limited by the low quality of such
synthetic datasets. Previous solutions have primarily focused on single PLM
settings, where synthetic datasets are typically restricted to specific
sub-spaces and often deviate from real-world distributions, leading to severe
distribution bias. To mitigate such bias, we propose FuseGen, a novel data
generation-based zero-shot learning framework that introduces a new criteria
for subset selection from synthetic datasets via utilizing multiple PLMs and
trained STMs. The chosen subset provides in-context feedback to each PLM,
enhancing dataset quality through iterative data generation. Trained STMs are
then used for sample re-weighting as well, further improving data quality.
Extensive experiments across diverse tasks demonstrate that FuseGen
substantially outperforms existing methods, highly effective in boosting STM
performance in a PLM-agnostic way. Code is provided in
this https URL

基于数据生成的零样本学习通过由预训练语言模型生成的合成数据集对小任务特定模型进行训练，虽然有效果，但往往受到合成数据集质量低的限制。本文提出了 FuseGen，一种基于数据生成的零样本学习框架，通过利用多个预训练语言模型和训练过的小任务特定模型，在合成数据集中引入了一种新的子集选择标准，从而提供了上下文反馈，通过迭代数据生成增强了数据集质量。本方法在各种任务上的广泛实验证明，相较于现有方法，FuseGen 显著提升了小任务特定模型的性能，且不依赖于预训练语言模型。代码可在提供的链接中获得。

FuseGen: 基于数据生成的零样本学习的 PLM 融合

FuseGen: PLM Fusion for Data-generation based Zero-shot Learning

Language bias is a critical issue in Visual Question Answering (VQA), where
models often exploit dataset biases for the final decision without considering
the image information. As a result, they suffer from performance drop on
out-of-distribution data and inadequate visual explanation. Based on
experimental analysis for existing robust VQA methods, we stress the language
bias in VQA that comes from two aspects, i.e., distribution bias and shortcut
bias. We further propose a new de-bias framework, Greedy Gradient Ensemble
(GGE), which combines multiple biased models for unbiased base model learning.
With the greedy strategy, GGE forces the biased models to over-fit the biased
data distribution in priority, thus makes the base model pay more attention to
examples that are hard to solve by biased models. The experiments demonstrate
that our method makes better use of visual information and achieves
state-of-the-art performance on diagnosing dataset VQA-CP without using extra
annotations.

本篇研究针对视觉问答 (VQA) 中的语言偏差问题进行了实验分析，提出了一种新的去偏差框架 GGE，该框架将多个有偏差的模型组合起来进行无偏差的基础模型学习，实现了更好地利用视觉信息，并在不使用额外注释的情况下，在 VQA-CP 数据集上达到了最先进的性能。

贪心梯度集成用于强健视觉问答

Greedy Gradient Ensemble for Robust Visual Question Answering

There has been increased interest in devising learning techniques that
combine unlabeled data with labeled data ? i.e. semi-supervised learning.
However, to the best of our knowledge, no study has been performed across
various techniques and different types and amounts of labeled and unlabeled
data. Moreover, most of the published work on semi-supervised learning
techniques assumes that the labeled and unlabeled data come from the same
distribution. It is possible for the labeling process to be associated with a
selection bias such that the distributions of data points in the labeled and
unlabeled sets are different. Not correcting for such bias can result in biased
function approximation with potentially poor performance. In this paper, we
present an empirical study of various semi-supervised learning techniques on a
variety of datasets. We attempt to answer various questions such as the effect
of independence or relevance amongst features, the effect of the size of the
labeled and unlabeled sets and the effect of noise. We also investigate the
impact of sample-selection bias on the semi-supervised learning techniques
under study and implement a bivariate probit technique particularly designed to
correct for such bias.

本文对多种半监督学习技术在不同数据集上进行实证研究，探讨标记数据和未标记数据的独立性、相关性、数量和噪声对学习效果的影响，同时针对样本选择偏差实现双变量 Probit 技术进行校正。