With the evolution of storage and communication protocols, ultra-low bitrate
image compression has become a highly demanding topic. However, existing
compression algorithms must sacrifice either consistency with the ground truth
or perceptual quality at ultra-low bitrate. In recent years, the rapid
development of the Large Multimodal Model (LMM) has made it possible to balance
these two goals. To solve this problem, this paper proposes a method called
Multimodal Image Semantic Compression (MISC), which consists of an LMM encoder
for extracting the semantic information of the image, a map encoder to locate
the region corresponding to the semantic, an image encoder generates an
extremely compressed bitstream, and a decoder reconstructs the image based on
the above information. Experimental results show that our proposed MISC is
suitable for compressing both traditional Natural Sense Images (NSIs) and
emerging AI-Generated Images (AIGIs) content. It can achieve optimal
consistency and perception results while saving 50% bitrate, which has strong
potential applications in the next generation of storage and communication. The
code will be released on this https URL

该研究提出了一种名为多模态图像语义压缩（MISC）的方法，采用大型多模态模型（LMM）来平衡传统自然感知图像和人工智能生成图像的压缩，实现了一致性和感知结果的优化，节省了 50％的比特率，并在存储和通信领域具有强大的应用潜力。