With the rapid advancement of Vision Language Models (VLMs), VLM-based Image
Quality Assessment (IQA) seeks to describe image quality linguistically to
align with human expression and capture the multifaceted nature of IQA tasks.
However, current methods are still far from practical usage. First, prior works
focus narrowly on specific sub-tasks or settings, which do not align with
diverse real-world applications. Second, their performance is sub-optimal due
to limitations in dataset coverage, scale, and quality. To overcome these
challenges, we introduce Depicted image Quality Assessment in the Wild
(DepictQA-Wild). Our method includes a multi-functional IQA task paradigm that
encompasses both assessment and comparison tasks, brief and detailed responses,
full-reference and non-reference scenarios. We introduce a
ground-truth-informed dataset construction approach to enhance data quality,
and scale up the dataset to 495K under the brief-detail joint framework.
Consequently, we construct a comprehensive, large-scale, and high-quality
dataset, named DQ-495K. We also retain image resolution during training to
better handle resolution-related quality issues, and estimate a confidence
score that is helpful to filter out low-quality responses. Experimental results
demonstrate that DepictQA-Wild significantly outperforms traditional
score-based methods, prior VLM-based IQA models, and proprietary GPT-4V in
distortion identification, instant rating, and reasoning tasks. Our advantages
are further confirmed by real-world applications including assessing the
web-downloaded images and ranking model-processed images. Datasets and codes
will be released in this https URL

基于 Depicted Image Quality Assessment in the Wild (DepictQA-Wild) 方法构建了一个多功能的图像质量评估任务范式，包括评估和比较任务，从而构建了一个全面、大规模且高质量的数据集 DQ-495K，结果显示 DepictQA-Wild 在失真识别、即时评分和推理任务中明显优于传统基于分数的方法、之前的基于 Vision Language Models 的图像质量评估模型和专有的 GPT-4V。

野外环境下的图像描述性质量评估

Descriptive Image Quality Assessment in the Wild

While abundant research has been conducted on improving high-level visual
understanding and reasoning capabilities of large multimodal models~(LMMs),
their visual quality assessment~(IQA) ability has been relatively
under-explored. Here we take initial steps towards this goal by employing the
two-alternative forced choice~(2AFC) prompting, as 2AFC is widely regarded as
the most reliable way of collecting human opinions of visual quality.
Subsequently, the global quality score of each image estimated by a particular
LMM can be efficiently aggregated using the maximum a posterior estimation.
Meanwhile, we introduce three evaluation criteria: consistency, accuracy, and
correlation, to provide comprehensive quantifications and deeper insights into
the IQA capability of five LMMs. Extensive experiments show that existing LMMs
exhibit remarkable IQA ability on coarse-grained quality comparison, but there
is room for improvement on fine-grained quality discrimination. The proposed
dataset sheds light on the future development of IQA models based on LMMs. The
codes will be made publicly available at this https URL

通过使用 2AFC 提示，评估了大型多模态模型（LMMs）的图像质量评估（IQA）能力，并引入了三个评估标准，结果显示现有的 LMMs 在粗粒度的质量比较上表现出较高的 IQA 能力，但在细粒度的质量判别方面仍有提升空间。

2AFC 大型多模态模型的图像质量评估

2AFC Prompting of Large Multimodal Models for Image Quality Assessment

Due to the subjective nature of image quality assessment (IQA), assessing
which image has better quality among a sequence of images is more reliable than
assigning an absolute mean opinion score for an image. Thus, IQA models are
evaluated by global correlation consistency (GCC) metrics like PLCC and SROCC,
rather than mean opinion consistency (MOC) metrics like MAE and MSE. However,
most existing methods adopt MOC metrics to define their loss functions, due to
the infeasible computation of GCC metrics during training. In this work, we
construct a novel loss function and network to exploit Global-correlation and
Mean-opinion Consistency, forming a GMC-IQA framework. Specifically, we propose
a novel GCC loss by defining a pairwise preference-based rank estimation to
solve the non-differentiable problem of SROCC and introducing a queue mechanism
to reserve previous data to approximate the global results of the whole data.
Moreover, we propose a mean-opinion network, which integrates diverse opinion
features to alleviate the randomness of weight learning and enhance the model
robustness. Experiments indicate that our method outperforms SOTA methods on
multiple authentic datasets with higher accuracy and generalization. We also
adapt the proposed loss to various networks, which brings better performance
and more stable training.

通过构建一个新的损失函数和网络，本研究提出了一个 GMC-IQA 框架，综合了全局相关性和均值一致性，通过定义基于偏好的排序估计来解决非可微问题，并引入了队列机制来近似整个数据集的全局结果。实验证明，我们的方法在多个真实数据集上的准确性和泛化能力上优于其他方法，并且适应了多种网络，取得了更好的性能和更稳定的训练。