No-reference image- and video-quality metrics are widely used in video
processing benchmarks. The robustness of learning-based metrics under video
attacks has not been widely studied. In addition to having success, attacks
that can be employed in video processing benchmarks must be fast and
imperceptible. This paper introduces an Invisible One-Iteration (IOI)
adversarial attack on no reference image and video quality metrics. We compared
our method alongside eight prior approaches using image and video datasets via
objective and subjective tests. Our method exhibited superior visual quality
across various attacked metric architectures while maintaining comparable
attack success and speed. We made the code available on GitHub.

本论文介绍一种针对无参考图像和视频质量度量的隐形一次迭代对抗性攻击方法，通过客观和主观测试，与八种先前方法进行比较，我们方法在各种被攻击的度量架构中表现出卓越的视觉质量，并同时保持可比较的攻击成功率和速度。

IOI：一种对无参考图像和视频质量测评方法的隐形一遍攻击

IOI: Invisible One-Iteration Adversarial Attack on No-Reference Image-  and Video-Quality Metrics

Open-domain generative dialogue systems have attracted considerable attention
over the past few years. Currently, how to automatically evaluate them, is
still a big challenge problem. As far as we know, there are three kinds of
automatic methods to evaluate the open-domain generative dialogue systems: (1)
Word-overlap-based metrics; (2) Embedding-based metrics; (3) Learning-based
metrics. Due to the lack of systematic comparison, it is not clear which kind
of metrics are more effective. In this paper, we will first measure
systematically all kinds of automatic evaluation metrics over the same
experimental setting to check which kind is best. Through extensive
experiments, the learning-based metrics are demonstrated that they are the most
effective evaluation metrics for open-domain generative dialogue systems.
Moreover, we observe that nearly all learning-based metrics depend on the
negative sampling mechanism, which obtains an extremely imbalanced and
low-quality dataset to train a score model. In order to address this issue, we
propose a novel and feasible learning-based metric that can significantly
improve the correlation with human judgments by using augmented POsitive
samples and valuable NEgative samples, called PONE. Extensive experiments
demonstrate that our proposed evaluation method significantly outperforms the
state-of-the-art learning-based evaluation methods, with an average correlation
improvement of 13.18%. In addition, we have publicly released the codes of our
proposed method and state-of-the-art baselines.

通过对三类自动评估方法的系统比较，本文认为基于学习的指标是评估开放领域生成对话系统最有效的方法。为了解决由于负采样机制导致的数据集不平衡和低质问题，本文提出了一种新的基于学习的评估指标 PONE，并经过实验证明，在使用增强的正样本和有价值的负样本后，可以显著提高与人类判断的相关性，平均相关性提高达 13.18％。