This paper studies a novel energy-based cooperative learning framework for
multi-domain image-to-image translation. The framework consists of four
components: descriptor, translator, style encoder, and style generator. The
descriptor is a multi-head energy-based model that represents a multi-domain
image distribution. The components of translator, style encoder, and style
generator constitute a diversified image generator. Specifically, given an
input image from a source domain, the translator turns it into a stylised
output image of the target domain according to a style code, which can be
inferred by the style encoder from a reference image or produced by the style
generator from a random noise. Since the style generator is represented as an
domain-specific distribution of style codes, the translator can provide a
one-to-many transformation (i.e., diversified generation) between source domain
and target domain. To train our framework, we propose a likelihood-based
multi-domain cooperative learning algorithm to jointly train the multi-domain
descriptor and the diversified image generator (including translator, style
encoder, and style generator modules) via multi-domain MCMC teaching, in which
the descriptor guides the diversified image generator to shift its probability
density toward the data distribution, while the diversified image generator
uses its randomly translated images to initialize the descriptor's Langevin
dynamics process for efficient sampling.

本文研究了一种新型一种基于能量的协作学习框架，用于多领域图像到图像的转换。该框架由描述符、翻译器、样式编码器和样式生成器四个组成部分组成，能够实现一个到多个的转换。该框架通过多领域 MCMC 教学共同训练多领域的描述符和多样化的图像生成器。