The one-shot talking-head generation learns to synthesize a talking-head
video with one source portrait image under the driving of same or different
identity video. Usually these methods require plane-based pixel transformations
via Jacobin matrices or facial image warps for novel poses generation. The
constraints of using a single image source and pixel displacements often
compromise the clarity of the synthesized images. Some methods try to improve
the quality of synthesized videos by introducing additional super-resolution
modules, but this will undoubtedly increase computational consumption and
destroy the original data distribution. In this work, we propose an adaptive
high-quality talking-head video generation method, which synthesizes
high-resolution video without additional pre-trained modules. Specifically,
inspired by existing super-resolution methods, we down-sample the one-shot
source image, and then adaptively reconstruct high-frequency details via an
encoder-decoder module, resulting in enhanced video clarity. Our method
consistently improves the quality of generated videos through a straightforward
yet effective strategy, substantiated by quantitative and qualitative
evaluations. The code and demo video are available on:
https://github.com/Songluchuan/AdaSR-TalkingHead/.

提出了一种自适应的高质量说话头视频生成方法，通过降采样源图像并利用编码解码模块自适应重构高频细节，以提高合成视频的清晰度。

自适应超分辨率用于一次性说话头生成

Adaptive Super Resolution For One-Shot Talking-Head Generation

In smartphones and compact cameras, the Image Signal Processor (ISP)
transforms the RAW sensor image into a human-readable sRGB image. Most popular
super-resolution methods depart from a sRGB image and upscale it further,
improving its quality. However, modeling the degradations in the sRGB domain is
complicated because of the non-linear ISP transformations. Despite this known
issue, only a few methods work directly with RAW images and tackle real-world
sensor degradations. We tackle blind image super-resolution in the RAW domain.
We design a realistic degradation pipeline tailored specifically for training
models with raw sensor data. Our approach considers sensor noise, defocus,
exposure, and other common issues. Our BSRAW models trained with our pipeline
can upscale real-scene RAW images and improve their quality. As part of this
effort, we also present a new DSLM dataset and benchmark for this task.

通过直接处理原始图像，设计了一个适用于训练原始传感器数据模型的逼真退化管道，以实现在原始领域盲目图像超分辨率，并提出了一个新的数据集和基准

BSRAW：提升盲目 RAW 图像超分辨率

BSRAW: Improving Blind RAW Image Super-Resolution

In this work, we introduce a challenging image restoration task, referred to
as SuperInpaint, which aims to reconstruct missing regions in low-resolution
images and generate completed images with arbitrarily higher resolutions. We
have found that this task cannot be effectively addressed by stacking
state-of-the-art super-resolution and image inpainting methods as they amplify
each other's flaws, leading to noticeable artifacts. To overcome these
limitations, we propose the detail-enhanced attentional implicit representation
(DEAR) that can achieve SuperInpaint with a single model, resulting in
high-quality completed images with arbitrary resolutions. Specifically, we use
a deep convolutional network to extract the latent embedding of an input image
and then enhance the high-frequency components of the latent embedding via an
adaptive high-pass filter. This leads to detail-enhanced semantic embedding. We
further feed the semantic embedding into an unmask-attentional module that
suppresses embeddings from ineffective masked pixels. Additionally, we extract
a pixel-wise importance map that indicates which pixels should be used for
image reconstruction. Given the coordinates of a pixel we want to reconstruct,
we first collect its neighboring pixels in the input image and extract their
detail-enhanced semantic embeddings, unmask-attentional semantic embeddings,
importance values, and spatial distances to the desired pixel. Then, we feed
all the above terms into an implicit representation and generate the color of
the specified pixel. To evaluate our method, we extend three existing datasets
for this new task and build 18 meaningful baselines using SOTA inpainting and
super-resolution methods. Extensive experimental results demonstrate that our
method outperforms all existing methods by a significant margin on four widely
used metrics.

通过引入细节增强的注意力隐式表示（DEAR）方法，该研究旨在解决低分辨率图像中的遗失区域恢复和生成任意高分辨率的图像，并通过使用单一模型在高质量完成的图像方面取得成功。