The emergence of pretrained models has significantly impacted from Natural
Language Processing (NLP) and Computer Vision to relational datasets.
Traditionally, these models are assessed through fine-tuned downstream tasks.
However, this raises the question of how to evaluate these models more
efficiently and more effectively. In this study, we explore a novel approach
where we leverage the meta features associated with each entity as a source of
worldly knowledge and employ entity representations from the models. We propose
using the consistency between these representations and the meta features as a
metric for evaluating pretrained models. Our method's effectiveness is
demonstrated across various domains, including models with relational datasets,
large language models and images models.

通过利用元特征和模型提取的实体表示之间的一致性作为衡量预训练模型的度量标准，本研究探索了一种评估预训练模型更有效且更高效的方法，该方法在包括有关关系数据集的模型、大型语言模型和图像模型的各个领域中得到了有效验证。

预先训练模型是否改善？一种基于多头后验的方法

Has Your Pretrained Model Improved? A Multi-head Posterior Based  Approach

Existing Cross Modal Hashing (CMH) methods are mainly designed for balanced
data, while imbalanced data with long-tail distribution is more general in
real-world. Several long-tail hashing methods have been proposed but they can
not adapt for multi-modal data, due to the complex interplay between labels and
individuality and commonality information of multi-modal data. Furthermore, CMH
methods mostly mine the commonality of multi-modal data to learn hash codes,
which may override tail labels encoded by the individuality of respective
modalities. In this paper, we propose LtCMH (Long-tail CMH) to handle
imbalanced multi-modal data. LtCMH firstly adopts auto-encoders to mine the
individuality and commonality of different modalities by minimizing the
dependency between the individuality of respective modalities and by enhancing
the commonality of these modalities. Then it dynamically combines the
individuality and commonality with direct features extracted from respective
modalities to create meta features that enrich the representation of tail
labels, and binaries meta features to generate hash codes. LtCMH significantly
outperforms state-of-the-art baselines on long-tail datasets and holds a better
(or comparable) performance on datasets with balanced labels.

本文提出了一种处理不平衡多模态数据的方法 LtCMH (Long-tail CMH), 使用自编码器挖掘不同模态的个性和共性，并将其与来自各模态的特征相结合产生元特征，此方法在长尾分布数据下显著优于现有算法，在平衡标签数据集上表现更好（或与之相当）