Learned image compression (LIC) methods have experienced significant progress
during recent years. However, these methods are primarily dedicated to
optimizing the rate-distortion (R-D) performance at medium and high bitrates (>
0.1 bits per pixel (bpp)), while research on extremely low bitrates is limited.
Besides, existing methods fail to explicitly explore the image structure and
texture components crucial for image compression, treating them equally
alongside uninformative components in networks. This can cause severe
perceptual quality degradation, especially under low-bitrate scenarios. In this
work, inspired by the success of pre-trained masked autoencoders (MAE) in many
downstream tasks, we propose to rethink its mask sampling strategy from
structure and texture perspectives for high redundancy reduction and
discriminative feature representation, further unleashing the potential of LIC
methods. Therefore, we present a dual-adaptive masking approach (DA-Mask) that
samples visible patches based on the structure and texture distributions of
original images. We combine DA-Mask and pre-trained MAE in masked image
modeling (MIM) as an initial compressor that abstracts informative semantic
context and texture representations. Such a pipeline can well cooperate with
LIC networks to achieve further secondary compression while preserving
promising reconstruction quality. Consequently, we propose a simple yet
effective masked compression model (MCM), the first framework that unifies MIM
and LIC end-to-end for extremely low-bitrate image compression. Extensive
experiments have demonstrated that our approach outperforms recent
state-of-the-art methods in R-D performance, visual quality, and downstream
applications, at very low bitrates. Our code is available at
this https URL

本文提出了一种新颖的双自适应掩膜 (DA-Mask) 方法，该方法结合了贴片结构和纹理分布，在极低比特率下有效压缩图像。同时，该研究还结合预先训练的掩膜自编码器 (MAE) 和 DA-Mask 以及 LIC 网络提出了一种基于贴片图像建模 (MIM) 的掩膜压缩模型 (MCM) ，该方法在 R-D 性能、可视质量和下游应用方面优于最新的同类研究。