Contrastive learning has considerably advanced the field of Image Quality
Assessment (IQA), emerging as a widely adopted technique. The core mechanism of
contrastive learning involves minimizing the distance between quality-similar
(positive) examples while maximizing the distance between quality-dissimilar
(negative) examples. Despite its successes, current contrastive learning
methods often neglect the importance of preserving the local manifold
structure. This oversight can result in a high degree of similarity among hard
examples within the feature space, thereby impeding effective differentiation
and assessment. To address this issue, we propose an innovative framework that
integrates local manifold learning with contrastive learning for No-Reference
Image Quality Assessment (NR-IQA). Our method begins by sampling multiple crops
from a given image, identifying the most visually salient crop. This crop is
then used to cluster other crops from the same image as the positive class,
while crops from different images are treated as negative classes to increase
inter-class distance. Uniquely, our approach also considers non-saliency crops
from the same image as intra-class negative classes to preserve their
distinctiveness. Additionally, we employ a mutual learning framework, which
further enhances the model's ability to adaptively learn and identify visual
saliency regions. Our approach demonstrates a better performance compared to
state-of-the-art methods in 7 standard datasets, achieving PLCC values of 0.942
(compared to 0.908 in TID2013) and 0.914 (compared to 0.894 in LIVEC).

我们提出了一种将局部流形学习与对比学习相结合的创新框架，用于无参考图像质量评估。该方法通过从给定图像中采样多个裁剪，识别出最具视觉显著性的裁剪，并将其他来自同一图像的裁剪作为正类进行聚类，将来自不同图像的裁剪作为负类以增加类间距离。与其他方法不同的是，我们的方法还将来自同一图像的非显著性裁剪作为类内负类以保持其独特性。此外，我们还采用了互相学习的框架，进一步增强了模型的自适应学习和视觉显著性区域识别能力。我们的方法在 7 个标准数据集中表现出更好的性能，达到了 0.942 的 PLCC 值（与 TID2013 的 0.908 相比）和 0.914 的 PLCC 值（与 LIVEC 的 0.894 相比）。

无参考图像质量评估的局部流形学习

Local Manifold Learning for No-Reference Image Quality Assessment

3D scene stylization aims at generating stylized images of the scene from
arbitrary novel views following a given set of style examples, while ensuring
consistency when rendered from different views. Directly applying methods for
image or video stylization to 3D scenes cannot achieve such consistency. Thanks
to recently proposed neural radiance fields (NeRF), we are able to represent a
3D scene in a consistent way. Consistent 3D scene stylization can be
effectively achieved by stylizing the corresponding NeRF. However, there is a
significant domain gap between style examples which are 2D images and NeRF
which is an implicit volumetric representation. To address this problem, we
propose a novel mutual learning framework for 3D scene stylization that
combines a 2D image stylization network and NeRF to fuse the stylization
ability of 2D stylization network with the 3D consistency of NeRF. We first
pre-train a standard NeRF of the 3D scene to be stylized and replace its color
prediction module with a style network to obtain a stylized NeRF. It is
followed by distilling the prior knowledge of spatial consistency from NeRF to
the 2D stylization network through an introduced consistency loss. We also
introduce a mimic loss to supervise the mutual learning of the NeRF style
module and fine-tune the 2D stylization decoder. In order to further make our
model handle ambiguities of 2D stylization results, we introduce learnable
latent codes that obey the probability distributions conditioned on the style.
They are attached to training samples as conditional inputs to better learn the
style module in our novel stylized NeRF. Experimental results demonstrate that
our method is superior to existing approaches in both visual quality and
long-range consistency.

本文提出了一种新的双向学习框架用于 3D 场景风格化，结合 2D 图像风格化网络和 NeRF，并引入了一种一致性损失函数来迁移 NeRF 的一致性先验知识到 2D 风格化网络中。实验结果表明，这种方法在视觉质量和长程一致性方面优于现有方法。

StylizedNeRF: 通过 2D-3D 相互学习将一致的 3D 场景风格化为 Stylized NeRF

StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D  Mutual Learning

Code generation aims to automatically generate a piece of code given an input
natural language utterance. Currently, among dominant models, it is treated as
a sequence-to-tree task, where a decoder outputs a sequence of actions
corresponding to the pre-order traversal of an Abstract Syntax Tree. However,
such a decoder only exploits the preorder traversal based preceding actions,
which are insufficient to ensure correct action predictions. In this paper, we
first throughly analyze the context modeling difference between neural code
generation models with different traversals based decodings (preorder traversal
vs breadth-first traversal), and then propose to introduce a mutual learning
framework to jointly train these models. Under this framework, we continuously
enhance both two models via mutual distillation, which involves synchronous
executions of two one-to-one knowledge transfers at each training step. More
specifically, we alternately choose one model as the student and the other as
its teacher, and require the student to fit the training data and the action
prediction distributions of its teacher. By doing so, both models can fully
absorb the knowledge from each other and thus could be improved simultaneously.
Experimental results and in-depth analysis on several benchmark datasets
demonstrate the effectiveness of our approach. We release our code at
this https URL

本文提出了一种互联互通学习框架，以解决在神经代码生成模型中不同遍历方法的编码差异，并利用两种模型的互相蒸馏技术来提高模型的准确性。通过实验数据的支持，该模型在几个基准测试数据集上表现出很高的效果。