Bayesian inference with deep generative prior has received considerable
interest for solving imaging inverse problems in many scientific and
engineering fields. The selection of the prior distribution is learned from,
and therefore an important representation learning of, available prior
measurements. The SA-Roundtrip, a novel deep generative prior, is introduced to
enable controlled sampling generation and identify the data's intrinsic
dimension. This prior incorporates a self-attention structure within a
bidirectional generative adversarial network. Subsequently, Bayesian inference
is applied to the posterior distribution in the low-dimensional latent space
using the Hamiltonian Monte Carlo with preconditioned Crank-Nicolson (HMC-pCN)
algorithm, which is proven to be ergodic under specific conditions. Experiments
conducted on computed tomography (CT) reconstruction with the MNIST and
TomoPhantom datasets reveal that the proposed method outperforms
state-of-the-art comparisons, consistently yielding a robust and superior point
estimator along with precise uncertainty quantification.

一种使用深度生成先验进行贝叶斯推断的新方法，该方法通过引入具有自注意力结构的双向生成对抗网络的 SA-Roundtrip 先验，在低维潜在空间中应用具有预条件 Crank-Nicolson 算法的 Hamiltonian Monte Carlo 对后验分布进行推断，以实现可控采样生成并确定数据的内在维度。在计算机断层扫描重构中进行的实验结果表明，该方法优于当前最先进的方法，始终产生精确的点估计并提供准确的不确定性量化。

基于 HMC-pCN 采样方法的带有 SA-Roundtrip 先验的贝叶斯成像反问题

Bayesian imaging inverse problem with SA-Roundtrip prior via HMC-pCN  sampler

We introduce MyStyle, a personalized deep generative prior trained with a few
shots of an individual. MyStyle allows to reconstruct, enhance and edit images
of a specific person, such that the output is faithful to the person's key
facial characteristics. Given a small reference set of portrait images of a
person (~100), we tune the weights of a pretrained StyleGAN face generator to
form a local, low-dimensional, personalized manifold in the latent space. We
show that this manifold constitutes a personalized region that spans latent
codes associated with diverse portrait images of the individual. Moreover, we
demonstrate that we obtain a personalized generative prior, and propose a
unified approach to apply it to various ill-posed image enhancement problems,
such as inpainting and super-resolution, as well as semantic editing. Using the
personalized generative prior we obtain outputs that exhibit high-fidelity to
the input images and are also faithful to the key facial characteristics of the
individual in the reference set. We demonstrate our method with fair-use images
of numerous widely recognizable individuals for whom we have the prior
knowledge for a qualitative evaluation of the expected outcome. We evaluate our
approach against few-shots baselines and show that our personalized prior,
quantitatively and qualitatively, outperforms state-of-the-art alternatives.

本文介绍了 MyStyle，这是一种使用少量照片训练的个性化深度生成先验的方法，其可以重建、增强和编辑特定人物的图像，生成的图像忠实于人物的关键面部特征。通过少量照片我们可以调整预训练的 StyleGAN 面部生成器的权重，形成一个本地的、低维的、个性化的流形空间，该流形空间包含与个人不同的各种肖像图片的潜在编码。此外，我们提出了一种统一的方法，将其应用于各种不适定的图像增强问题，例如修复、超分辨率和语义编辑，并取得了优异的定量和定性水平。

MyStyle: 个性化生成先验

MyStyle: A Personalized Generative Prior

Learning a good image prior is a long-term goal for image restoration and
manipulation. While existing methods like deep image prior (DIP) capture
low-level image statistics, there are still gaps toward an image prior that
captures rich image semantics including color, spatial coherence, textures, and
high-level concepts. This work presents an effective way to exploit the image
prior captured by a generative adversarial network (GAN) trained on large-scale
natural images. As shown in Fig.1, the deep generative prior (DGP) provides
compelling results to restore missing semantics, e.g., color, patch,
resolution, of various degraded images. It also enables diverse image
manipulation including random jittering, image morphing, and category transfer.
Such highly flexible restoration and manipulation are made possible through
relaxing the assumption of existing GAN-inversion methods, which tend to fix
the generator. Notably, we allow the generator to be fine-tuned on-the-fly in a
progressive manner regularized by feature distance obtained by the
discriminator in GAN. We show that these easy-to-implement and practical
changes help preserve the reconstruction to remain in the manifold of nature
image, and thus lead to more precise and faithful reconstruction for real
images. Code is available at
this https URL

本文介绍了一种有效利用生成对抗网络 (DGP) 捕获的图像先验的方法，它可以恢复各种退化图像中缺失的语义信息，还可以进行多样化的图像操作，通过松弛现有的 GAN 反演方法的假设，允许生成器以渐进的方式进行微调，并在 GAN 中的鉴别器处获得的特征距离作为正则项。这些易于实现和实用的改变有助于保持重构，使其保持在自然图像流形中，从而可以更准确、更忠实地重构真实图像。