Retrieval-based augmentations that aim to incorporate knowledge from an
external database into language models have achieved great success in various
knowledge-intensive (KI) tasks, such as question-answering and text generation.
However, integrating retrievals in non-knowledge-intensive (NKI) tasks, such as
text classification, is still challenging. Existing works focus on
concatenating retrievals to inputs as context to form the prompt-based inputs.
Unfortunately, such methods require language models to have the capability to
handle long texts. Besides, inferring such concatenated data would also consume
a significant amount of computational resources.
To solve these challenges, we propose \textbf{ReFusion} in this paper, a
computation-efficient \textbf{Re}trieval representation \textbf{Fusion} with
neural architecture search. The main idea is to directly fuse the retrieval
representations into the language models. Specifically, we first propose an
online retrieval module that retrieves representations of similar sentences.
Then, we present a retrieval fusion module including two effective ranking
schemes, i.e., reranker-based scheme and ordered-mask-based scheme, to fuse the
retrieval representations with hidden states. Furthermore, we use Neural
Architecture Search (NAS) to seek the optimal fusion structure across different
layers. Finally, we conduct comprehensive experiments, and the results
demonstrate our ReFusion can achieve superior and robust performance on various
NKI tasks.

在本文中，我们提出了一种计算高效的检索表示融合方法 ReFusion，通过神经架构搜索，直接将检索表示融入到语言模型中，来解决将外部数据库中的知识纳入到非知识密集型任务中的挑战。实验证明，我们的 ReFusion 在各种非知识密集型任务上实现了卓越且鲁棒的性能。

提升自然语言理解的计算高效检索表示融合

Improving Natural Language Understanding with Computation-Efficient  Retrieval Representation Fusion

This work aims to improve the applicability of diffusion models in realistic
image restoration. Specifically, we enhance the diffusion model in several
aspects such as network architecture, noise level, denoising steps, training
image size, and optimizer/scheduler. We show that tuning these hyperparameters
allows us to achieve better performance on both distortion and perceptual
scores. We also propose a U-Net based latent diffusion model which performs
diffusion in a low-resolution latent space while preserving high-resolution
information from the original input for the decoding process. Compared to the
previous latent-diffusion model which trains a VAE-GAN to compress the image,
our proposed U-Net compression strategy is significantly more stable and can
recover highly accurate images without relying on adversarial optimization.
Importantly, these modifications allow us to apply diffusion models to various
image restoration tasks, including real-world shadow removal, HR
non-homogeneous dehazing, stereo super-resolution, and bokeh effect
transformation. By simply replacing the datasets and slightly changing the
noise network, our model, named Refusion, is able to deal with large-size
images (e.g., 6000 x 4000 x 3 in HR dehazing) and produces good results on all
the above restoration problems. Our Refusion achieves the best perceptual
performance in the NTIRE 2023 Image Shadow Removal Challenge and wins 2nd place
overall.

本研究旨在提高扩散模型在逼真图像修复中的适用性，特别是在网络架构、噪声水平、降噪步骤、训练图像大小和优化器 / 调度器等多个方面进行改进。我们展示了调节这些超参数可以在失真和感知得分上实现更好的性能，并提出了基于 U-Net 的潜在扩散模型。与以前的潜在扩散模型相比，我们提出的 U-Net 压缩策略更稳定，而且不依赖敌对性优化就能恢复高度准确的图像。我们的模型 Refusion 能够处理大尺寸图像，包括真实世界的阴影去除、HR 不均匀去雾、立体超分辨率和景深效果转换，并在 NTIRE 2023 图像阴影去除挑战赛中获得最佳感知性能和第二名。