Recently, large-scale pre-trained vision-language models have presented
benefits for alleviating class imbalance in long-tailed recognition. However,
the long-tailed data distribution can corrupt the representation space, where
the distance between head and tail categories is much larger than the distance
between two tail categories. This uneven feature space distribution causes the
model to exhibit unclear and inseparable decision boundaries on the uniformly
distributed test set, which lowers its performance. To address these
challenges, we propose the uniformly category prototype-guided vision-language
framework to effectively mitigate feature space bias caused by data imbalance.
Especially, we generate a set of category prototypes uniformly distributed on a
hypersphere. Category prototype-guided mechanism for image-text matching makes
the features of different classes converge to these distinct and uniformly
distributed category prototypes, which maintain a uniform distribution in the
feature space, and improve class boundaries. Additionally, our proposed
irrelevant text filtering and attribute enhancement module allows the model to
ignore irrelevant noisy text and focus more on key attribute information,
thereby enhancing the robustness of our framework. In the image recognition
fine-tuning stage, to address the positive bias problem of the learnable
classifier, we design the class feature prototype-guided classifier, which
compensates for the performance of tail classes while maintaining the
performance of head classes. Our method outperforms previous vision-language
methods for long-tailed learning work by a large margin and achieves
state-of-the-art performance.

研究论文提出了一种基于统一类别原型导向的视觉语言框架，用于有效缓解数据不平衡引起的特征空间偏差，并通过去除无关噪声文本和增强关键属性信息的模块提高模型的鲁棒性，同时通过类特征原型导向分类器解决学习分类器的正向偏差问题，从而在长尾学习中实现了比之前方法更优越的性能。

长尾识别的均匀分布类别原型引导视觉语言框架

Uniformly Distributed Category Prototype-Guided Vision-Language  Framework for Long-Tail Recognition

Extracting image semantics effectively and assigning corresponding labels to
multiple objects or attributes for natural images is challenging due to the
complex scene contents and confusing label dependencies. Recent works have
focused on modeling label relationships with graph and understanding object
regions using class activation maps (CAM). However, these methods ignore the
complex intra- and inter-category relationships among specific semantic
features, and CAM is prone to generate noisy information. To this end, we
propose a novel semantic-aware dual contrastive learning framework that
incorporates sample-to-sample contrastive learning (SSCL) as well as
prototype-to-sample contrastive learning (PSCL). Specifically, we leverage
semantic-aware representation learning to extract category-related local
discriminative features and construct category prototypes. Then based on SSCL,
label-level visual representations of the same category are aggregated
together, and features belonging to distinct categories are separated.
Meanwhile, we construct a novel PSCL module to narrow the distance between
positive samples and category prototypes and push negative samples away from
the corresponding category prototypes. Finally, the discriminative label-level
features related to the image content are accurately captured by the joint
training of the above three parts. Experiments on five challenging large-scale
public datasets demonstrate that our proposed method is effective and
outperforms the state-of-the-art methods. Code and supplementary materials are
released on this https URL

通过引入样本对间对比学习和原型 - 样本对比学习，提出了一种新颖的语义感知双对比学习框架，通过联合训练三个模块，准确捕捉与图像内容相关的判别性标签级特征，并在五个具有挑战性的大规模公共数据集上实验证明了该方法的有效性优于现有技术。

多标签图像分类的语义感知双对比学习

Semantic-Aware Dual Contrastive Learning for Multi-label Image  Classification

Object detection and recognition are fundamental functions underlying the
success of species. Because the appearance of an object exhibits a large
variability, the brain has to group these different stimuli under the same
object identity, a process of generalization. Does the process of
generalization follow some general principles or is it an ad-hoc
"bag-of-tricks"? The Universal Law of Generalization provided evidence that
generalization follows similar properties across a variety of species and
tasks. Here we test the hypothesis that the internal representations underlying
generalization reflect the natural properties of object detection and
recognition in our environment rather than the specifics of the system solving
these problems. By training a deep-neural-network with images of "clear" and
"camouflaged" animals, we found that with a proper choice of category
prototypes, the generalization functions are monotone decreasing, similar to
the generalization functions of biological systems. Our findings support the
hypothesis of the study.

本文通过训练深度神经网络，并选取合适的类别原型，在动物的辨识中测试其广义化能力，结果显示其泛化函数随着类别数量的增加而逐渐递减，这一发现支持了研究的假设，即广义化过程反映了物体检测和识别的自然特性。