The surrogate loss of variational autoencoders (VAEs) poses various
challenges to their training, inducing the imbalance between task fitting and
representation inference. To avert this, the existing strategies for VAEs focus
on adjusting the tradeoff by introducing hyperparameters, deriving a tighter
bound under some mild assumptions, or decomposing the loss components per
certain neural settings. VAEs still suffer from uncertain tradeoff learning.We
propose a novel evolutionary variational autoencoder (eVAE) building on the
variational information bottleneck (VIB) theory and integrative evolutionary
neural learning. eVAE integrates a variational genetic algorithm into VAE with
variational evolutionary operators including variational mutation, crossover,
and evolution. Its inner-outer-joint training mechanism synergistically and
dynamically generates and updates the uncertain tradeoff learning in the
evidence lower bound (ELBO) without additional constraints. Apart from learning
a lossy compression and representation of data under the VIB assumption, eVAE
presents an evolutionary paradigm to tune critical factors of VAEs and deep
neural networks and addresses the premature convergence and random search
problem by integrating evolutionary optimization into deep learning.
Experiments show that eVAE addresses the KL-vanishing problem for text
generation with low reconstruction loss, generates all disentangled factors
with sharp images, and improves the image generation quality,respectively. eVAE
achieves better reconstruction loss, disentanglement, and generation-inference
balance than its competitors.

该研究提出了一种将遗传算法和变分自编码器相结合的进化变分自编码器方法，通过动态生成和更新不确定性权衡学习，解决了现有方法中面临的问题，包括 KL 消失问题和基于深度学习中的早期收敛和随机搜索问题。实验结果表明，与竞争方法相比，eVAE 在文本生成、图像生成质量等方面均有显著改善。