Image restoration is a fundamental problem that involves recovering a
high-quality clean image from its degraded observation. All-In-One image
restoration models can effectively restore images from various types and levels
of degradation using degradation-specific information as prompts to guide the
restoration model. In this work, we present the first approach that uses
human-written instructions to guide the image restoration model. Given natural
language prompts, our model can recover high-quality images from their degraded
counterparts, considering multiple degradation types. Our method, InstructIR,
achieves state-of-the-art results on several restoration tasks including image
denoising, deraining, deblurring, dehazing, and (low-light) image enhancement.
InstructIR improves +1dB over previous all-in-one restoration methods.
Moreover, our dataset and results represent a novel benchmark for new research
on text-guided image restoration and enhancement. Our code, datasets and models
are available at: this https URL

使用人类编写的指令指导图像修复模型，InstructIR 能从多种退化类型的图像中恢复高质量图像，包括图像去噪、去雨、去模糊、去雾和低光图像增强等多个恢复任务，并在这些任务上实现了最新的结果，此方法还提供了一个新的基准，用于文本指导的图像修复和增强的研究。

遵循人类指令的高质量图像修复

High-Quality Image Restoration Following Human Instructions

We study the problem of completing various visual document understanding
(VDU) tasks, e.g., question answering and information extraction, on real-world
documents through human-written instructions. To this end, we propose
InstructDoc, the first large-scale collection of 30 publicly available VDU
datasets, each with diverse instructions in a unified format, which covers a
wide range of 12 tasks and includes open document types/formats. Furthermore,
to enhance the generalization performance on VDU tasks, we design a new
instruction-based document reading and understanding model, InstructDr, that
connects document images, image encoders, and large language models (LLMs)
through a trainable bridging module. Experiments demonstrate that InstructDr
can effectively adapt to new VDU datasets, tasks, and domains via given
instructions and outperforms existing multimodal LLMs and ChatGPT without
specific training.

我们研究通过人工编写的指令，在真实世界的文档上完成各种视觉文档理解（VDU）任务，例如问答和信息提取。为此，我们提出了 InstructDoc，这是第一个包含 30 个公开可用的 VDU 数据集的大规模集合，每个数据集都有统一格式的多样指令，涵盖了 12 项不同任务并包括多种文档类型 / 格式。此外，为了提高 VDU 任务的泛化性能，我们设计了一种新的基于指令的文档阅读和理解模型 InstructDr，通过可训练的桥接模块连接了文档图像，图像编码器和大型语言模型（LLMs）。实验证明，通过给定的指令，InstructDr 能够有效适应新的 VDU 数据集、任务和领域，且胜过现有的多模态 LLMs 和 ChatGPT 无需特定训练。