In the field of neural data compression, the prevailing focus has been on
optimizing algorithms for either classical distortion metrics, such as PSNR or
SSIM, or human perceptual quality. With increasing amounts of data consumed by
machines rather than humans, a new paradigm of machine-oriented
compression$\unicode{x2013}$which prioritizes the retention of features salient
for machine perception over traditional human-centric
criteria$\unicode{x2013}$has emerged, creating several new challenges to the
development, evaluation, and deployment of systems utilizing lossy compression.
In particular, it is unclear how different approaches to lossy compression will
affect the performance of downstream machine perception tasks. To address this
under-explored area, we evaluate various perception
models$\unicode{x2013}$including image classification, image segmentation,
speech recognition, and music source separation$\unicode{x2013}$under severe
lossy compression. We utilize several popular codecs spanning conventional,
neural, and generative compression architectures. Our results indicate three
key findings: (1) using generative compression, it is feasible to leverage
highly compressed data while incurring a negligible impact on machine
perceptual quality; (2) machine perceptual quality correlates strongly with
deep similarity metrics, indicating a crucial role of these metrics in the
development of machine-oriented codecs; and (3) using lossy compressed
datasets, (e.g. ImageNet) for pre-training can lead to counter-intuitive
scenarios where lossy compression increases machine perceptual quality rather
than degrading it. To encourage engagement on this growing area of research,
our code and experiments are available at:
this https URL

神经数据压缩中，以机器为导向的压缩方式对于机器感知任务具有可行性，并且与深度相似度度量相关，在机器感知质量的发展中扮演重要角色。

机器感知质量：评估严重有损压缩对音频和图像模型的影响

Machine Perceptual Quality: Evaluating the Impact of Severe Lossy  Compression on Audio and Image Models

With neural networks growing deeper and feature maps growing larger, limited
communication bandwidth with external memory (or DRAM) and power constraints
become a bottleneck in implementing network inference on mobile and edge
devices. In this paper, we propose an end-to-end differentiable bandwidth
efficient neural inference method with the activation compressed by neural data
compression method. Specifically, we propose a transform-quantization-entropy
coding pipeline for activation compression with symmetric exponential Golomb
coding and a data-dependent Gaussian entropy model for arithmetic coding.
Optimized with existing model quantization methods, low-level task of image
compression can achieve up to 19x bandwidth reduction with 6.21x energy saving.

我们提出了一种端到端可微分的带宽高效神经推理方法，通过神经数据压缩方法对激活进行压缩，可实现高达 19 倍的带宽减少和 6.21 倍的能量节省。

神经图像压缩的带宽有效推理

Bandwidth-efficient Inference for Neural Image Compression

Rate-distortion (R-D) function, a key quantity in information theory,
characterizes the fundamental limit of how much a data source can be compressed
subject to a fidelity criterion, by any compression algorithm. As researchers
push for ever-improving compression performance, establishing the R-D function
of a given data source is not only of scientific interest, but also sheds light
on the possible room for improving compression algorithms. Previous work on
this problem relied on distributional assumptions on the data source (Gibson,
2017) or only applied to discrete data (Blahut, 1972; Arimoto, 1972). By
contrast, this paper makes the first attempt at an algorithm for sandwiching
the R-D function of a general (not necessarily discrete) source requiring only
i.i.d. data samples. We estimate R-D sandwich bounds for a variety of
artificial and real-world data sources, in settings far beyond the feasibility
of any known method, and shed light on the optimality of neural data
compression (Ball\'e et al., 2021; Yang et al., 2022). Our R-D upper bound on
natural images indicates theoretical room for improving state-of-the-art image
compression methods by at least one dB in PSNR at various bitrates. Our data
and code can be found at this https URL

通过一种新的算法，本文提出了第一个普遍数据源（不一定是离散数据）的 R-D 函数的夹逼界算法，并在多个数据源上估计了 R-D 夹逼界，说明了神经数据压缩的优越性，并发现了改进图像压缩方法的潜在空间。

基于经验的三明治上下界率失真函数

Towards Empirical Sandwich Bounds on the Rate-Distortion Function

Neural data compression has been shown to outperform classical methods in
terms of $RD$ performance, with results still improving rapidly. At a high
level, neural compression is based on an autoencoder that tries to reconstruct
the input instance from a (quantized) latent representation, coupled with a
prior that is used to losslessly compress these latents. Due to limitations on
model capacity and imperfect optimization and generalization, such models will
suboptimally compress test data in general. However, one of the great strengths
of learned compression is that if the test-time data distribution is known and
relatively low-entropy (e.g. a camera watching a static scene, a dash cam in an
autonomous car, etc.), the model can easily be finetuned or adapted to this
distribution, leading to improved $RD$ performance. In this paper we take this
concept to the extreme, adapting the full model to a single video, and sending
model updates (quantized and compressed using a parameter-space prior) along
with the latent representation. Unlike previous work, we finetune not only the
encoder/latents but the entire model, and - during finetuning - take into
account both the effect of model quantization and the additional costs incurred
by sending the model updates. We evaluate an image compression model on
I-frames (sampled at 2 fps) from videos of the Xiph dataset, and demonstrate
that full-model adaptation improves $RD$ performance by ~1 dB, with respect to
encoder-only finetuning.

本文介绍一种基于自动编码器和潜变量表示的神经压缩方法，并探讨了在训练后适应单个视频、发送模型更新以及重新调整参数等方面的改进措施。结果表明，相较于仅调整编码器的方式，对整个模型进行调整可以提高图像在视频中的压缩性能约 1 dB。