Image quality assessment often relies on raw opinion scores provided by
subjects in subjective experiments, which can be noisy and unreliable. To
address this issue, postprocessing procedures such as ITU-R BT.500, ITU-T
P.910, and ITU-T P.913 have been standardized to clean up the original opinion
scores. These methods use annotator-based statistical priors, but they do not
take into account extensive information about the image itself, which limits
their performance in less annotated scenarios. Generally speaking, image
quality datasets usually contain similar scenes or distortions, and it is
inevitable for subjects to compare images to score a reasonable score when
scoring. Therefore, In this paper, we proposed Subjective Image Quality Score
Preprocessing Method perceptual similarity Subjective Preprocessing (PSP),
which exploit the perceptual similarity between images to alleviate subjective
bias in less annotated scenarios. Specifically, we model subjective scoring as
a conditional probability model based on perceptual similarity with previously
scored images, called subconscious reference scoring. The reference images are
stored by a neighbor dictionary, which is obtained by a normalized vector
dot-product based nearest neighbor search of the images' perceptual depth
features. Then the preprocessed score is updated by the exponential moving
average (EMA) of the subconscious reference scoring, called similarity
regularized EMA. Our experiments on multiple datasets (LIVE, TID2013, CID2013)
show that this method can effectively remove the bias of the subjective scores.
Additionally, Experiments prove that the Preprocesed dataset can improve the
performance of downstream IQA tasks very well.

我们提出了一种名为感知相似性主观预处理（PSP）的主观图像质量评分预处理方法，利用图像之间的感知相似性减轻了较少标注情景下的主观偏倚，并证明预处理的数据集可以很好地提高下游图像质量评估任务的性能。

基于感知相似性的主观图像质量评分预处理方法

Beyond MOS: Subjective Image Quality Score Preprocessing Method Based on  Perceptual Similarity

Recently, widespread 3D graphics (e.g., point clouds and meshes) have drawn
considerable efforts from academia and industry to assess their perceptual
quality by conducting subjective experiments. However, lacking a handy software
for 3D subjective experiments complicates the construction of 3D graphics
quality assessment datasets, thus hindering the prosperity of relevant fields.
In this paper, we develop a powerful platform with which users can flexibly
design their 3D subjective methodologies and build high-quality datasets,
easing a broad spectrum of 3D graphics subjective quality study. To accurately
illustrate the perceptual quality differences of 3D stimuli, our software can
simultaneously render the source stimulus and impaired stimulus and allows both
stimuli to respond synchronously to viewer interactions. Compared with amateur
3D visualization tool-based or image/video rendering-based schemes, our
approach embodies typical 3D applications while minimizing cognitive overload
during subjective experiments. We organized a subjective experiment involving
40 participants to verify the validity of the proposed software. Experimental
analyses demonstrate that subjective tests on our software can produce
reasonable subjective quality scores of 3D models. All resources in this paper
can be found at this https URL

通过使用一款强大的软件，我们可以设计灵活的三维主观研究方法并构建高质量的数据集，从而准确展示三维刺激的感知质量差异，并通过实验结果证明该软件可以产生合理的三维模型主观质量评分。

基于屏幕的 3D 主观实验软件

Screen-based 3D Subjective Experiment Software

In this paper we consider how to automatically create pleasing photo collages
created by placing a set of images on a limited canvas area. The task is
formulated as an optimization problem. Differently from existing
state-of-the-art approaches, we here exploit subjective experiments to model
and learn pleasantness from user preferences. To this end, we design an
experimental framework for the identification of the criteria that need to be
taken into account to generate a pleasing photo collage. Five different
thematic photo datasets are used to create collages using state-of-the-art
criteria. A first subjective experiment where several subjects evaluated the
collages, emphasizes that different criteria are involved in the subjective
definition of pleasantness. We then identify new global and local criteria and
design algorithms to quantify them. The relative importance of these criteria
are automatically learned by exploiting the user preferences, and new collages
are generated. To validate our framework, we performed several psycho-visual
experiments involving different users. The results shows that the proposed
framework allows to learn a novel computational model which effectively encodes
an inter-user definition of pleasantness. The learned definition of
pleasantness generalizes well to new photo datasets of different themes and
sizes not used in the learning. Moreover, compared with two state of the art
approaches, the collages created using our framework are preferred by the
majority of the users.

本文讨论如何自动化地生成美观的照片拼贴画，利用主观实验来学习和建模用户偏好，设计了一个实验框架来识别生成美观照片拼贴画需要考虑的标准，使用五个不同的主题照片数据集创建拼贴画，并通过考虑到用户对它们的偏好来学习新的全局和局部标准。经过多次心理视觉实验的验证，发现使用该框架创建的拼贴画与现有的两种方法相比，被多数用户更喜欢。