A multi-layer deep Gaussian process (DGP) model is a hierarchical composition of GP models with a greater expressive power. Exact DGP inference is intractable, which has motivated the recent development of deterministic and stochastic approximation methods. Unfortunately, the deterministic approximation methods yield a biased posterior belief while the stochastic one is computationally costly. This paper presents an implicit posterior variational inference (IPVI) framework for DGPs that can ideally recover an unbiased posterior belief and still preserve time efficiency. Inspired by generative adversarial networks, our IPVI framework achieves this by casting the DGP inference problem as a two-player game in which a Nash equilibrium, interestingly, coincides with an unbiased posterior belief. This consequently inspires us to devise a best-response dynamics algorithm to search for a Nash equilibrium (i.e., an unbiased posterior belief). Empirical evaluation shows that IPVI outperforms the state-of-the-art approximation methods for DGPs.

本文提出了一种基于隐式后验变分推断 (IPVI) 框架的多层深高斯过程模型，能够理想地恢复无偏后验信念，并仍保持时间效率高，该框架通过将 DGP 推断问题建模为一个两个玩家博弈，在该博弈平衡点处获得无偏后验信念，并设计最佳响应动态算法来寻找博弈均衡点，得到的实证表明 IPVI 在 DGPs 的近似方法中优于现有技术水平。

深高斯过程的隐式后验变分推断