Multi-modal Large Language Models (MLLMs) have a significant impact on
various tasks, due to their extensive knowledge and powerful perception and
generation capabilities. However, it still remains an open research problem on
applying MLLMs to low-level vision tasks. In this paper, we present a simple
MLLM-based Image Restoration framework to address this gap, namely Multi-modal
Large Language Model based Restoration Assistant (LLMRA). We exploit the
impressive capabilities of MLLMs to obtain the degradation information for
universal image restoration. By employing a pretrained multi-modal large
language model and a vision language model, we generate text descriptions and
encode them as context embedding with degradation information for the degraded
image. Through the proposed Context Enhance Module (CEM) and Degradation
Context based Transformer Network (DC-former), we integrate these context
embedding into the restoration network, contributing to more accurate and
adjustable image restoration. Based on the dialogue with the users, our method
leverages image degradation priors from MLLMs, providing low-level attributes
descriptions of the input low-quality images and the restored high-quality
images simultaneously. Extensive experiments demonstrate the superior
performance of our LLMRA in universal image restoration tasks.

我们提出了一种基于多模态大语言模型（MLLM）的图像修复框架，名为多模态大语言模型修复助理（LLMRA），通过利用 MLLM 的印象能力来获取通用图像修复的降级信息，并通过 Context Enhance 模块和 Degradation Context based Transformer Network 将这些上下文嵌入整合到修复网络中，提供了输入低质量图像和恢复后高质量图像的低级属性描述。大量实验证明了 LLMRA 在通用图像修复任务中的卓越性能。