Recently, large-scale language-image generative models have gained widespread
attention and many works have utilized generated data from these models to
further enhance the performance of perception tasks. However, not all generated
data can positively impact downstream models, and these methods do not
thoroughly explore how to better select and utilize generated data. On the
other hand, there is still a lack of research oriented towards active learning
on generated data. In this paper, we explore how to perform active learning
specifically for generated data in the long-tailed instance segmentation task.
Subsequently, we propose BSGAL, a new algorithm that online estimates the
contribution of the generated data based on gradient cache. BSGAL can handle
unlimited generated data and complex downstream segmentation tasks effectively.
Experiments show that BSGAL outperforms the baseline approach and effectually
improves the performance of long-tailed segmentation. Our code can be found at
this https URL

本文探讨了如何在长尾实例分割任务中专门对生成数据执行主动学习，并提出了一种新的基于梯度缓存的算法 BSGAL，可以有效处理无限的生成数据和复杂的下游分割任务。实验证明，BSGAL 优于基线方法，有效提高了长尾分割的性能。

长尾实例分割的生成式主动学习

Generative Active Learning for Long-tailed Instance Segmentation

Diffusion models have shown their effectiveness in generation tasks by
well-approximating the underlying probability distribution. However, diffusion
models are known to suffer from an amplified inherent bias from the training
data in terms of fairness. While the sampling process of diffusion models can
be controlled by conditional guidance, previous works have attempted to find
empirical guidance to achieve quantitative fairness. To address this
limitation, we propose a fairness-aware sampling method called
\textit{attribute switching} mechanism for diffusion models. Without additional
training, the proposed sampling can obfuscate sensitive attributes in generated
data without relying on classifiers. We mathematically prove and experimentally
demonstrate the effectiveness of the proposed method on two key aspects: (i)
the generation of fair data and (ii) the preservation of the utility of the
generated data.

通过使用属性交换机制的公平度感知采样方法，扩散模型在生成公平数据和保持生成数据效用两个关键方面具有有效性。

扩散模型中的公平采样通过切换机制

Fair Sampling in Diffusion Models through Switching Mechanism

Benefiting from prompt tuning, recent years have witnessed the promising
performance of pre-trained vision-language models, e.g., CLIP, on versatile
downstream tasks. In this paper, we focus on a particular setting of learning
adaptive prompts on the fly for each test sample from an unseen new domain,
which is known as test-time prompt tuning (TPT). Existing TPT methods typically
rely on data augmentation and confidence selection. However, conventional data
augmentation techniques, e.g., random resized crops, suffers from the lack of
data diversity, while entropy-based confidence selection alone is not
sufficient to guarantee prediction fidelity. To address these issues, we
propose a novel TPT method, named DiffTPT, which leverages pre-trained
diffusion models to generate diverse and informative new data. Specifically, we
incorporate augmented data by both conventional method and pre-trained stable
diffusion to exploit their respective merits, improving the models ability to
adapt to unknown new test data. Moreover, to ensure the prediction fidelity of
generated data, we introduce a cosine similarity-based filtration technique to
select the generated data with higher similarity to the single test sample. Our
experiments on test datasets with distribution shifts and unseen categories
demonstrate that DiffTPT improves the zero-shot accuracy by an average of
5.13\% compared to the state-of-the-art TPT method. Our code and models will be
publicly released.

使用 DiffTPT 方法通过利用预训练扩散模型生成多样且信息丰富的新数据，结合常规方法和预训练稳定扩散的增强数据，选择与单个测试样本更相似的生成数据以提高预测准确度，证明 DiffTPT 方法能够提高零样本准确率。