We present MixNMatch, a conditional generative model that learns to disentangle and encode background, object pose, shape, and texture from real images with minimal supervision, for mix-and-match image generation. We build upon FineGAN, an unconditional generative model, to learn the desired disentanglement and image generator, and leverage adversarial joint image-code distribution matching to learn the latent factor encoders. MixNMatch requires bounding boxes during training to model background, but requires no other supervision. Through extensive experiments, we demonstrate MixNMatch's ability to accurately disentangle, encode, and combine multiple factors for mix-and-match image generation, including sketch2color, cartoon2img, and img2gif applications. Our code/models/demo can be found at https://github.com/Yuheng-Li/MixNMatch

本研究提出了 MixNMatch，一种有条件的生成模型，可以在最小监督的情况下学习对背景、物体姿态、形状和纹理进行解耦和编码，并用于图像生成。通过与 FineGAN 结合，我们实现了所需的解耦和图像生成器，并利用对抗性的联合图像-代码分布匹配来学习潜在因素编码器。MixNMatch 在训练期间需要边界框来建模背景，但不需要其他监督。通过大量实验，我们展示了 MixNMatch 在多个图像生成应用中准确地解耦、编码和组合多个因素的能力，包括草图转彩色、卡通转图像和图像转 GIF。

MixNMatch: 条件图像生成的多因素解缠和编码