Existing Bayesian models, especially nonparametric Bayesian methods, rely heavily on specially conceived priors to incorporate domain knowledge for discovering improved latent representations. While priors can affect posterior distributions through Bayes' theorem, imposing posterior regularization is arguably more direct and in some cases can be more natural and easier. In this paper, we present regularized Bayesian inference (RegBayes), a computational framework to perform posterior inference with a convex regularization on the desired post-data posterior distributions. RegBayes covers both directed Bayesian networks and undirected Markov networks whose Bayesian formulation results in hybrid chain graph models. When the convex regularization is induced from a linear operator on the posterior distributions, RegBayes can be solved with convex analysis theory. Furthermore, we present two concrete examples of RegBayes, infinite latent support vector machines (iLSVM) and multi-task infinite latent support vector machines (MT-iLSVM), which explore the large-margin idea in combination with a nonparametric Bayesian model for discovering predictive latent features for classification and multi-task learning, respectively. We present efficient inference methods and report empirical studies on several benchmark datasets, which appear to demonstrate the merits inherited from both large-margin learning and Bayesian nonparametrics. Such results were not available until now, and contribute to push forward the interface between these two important subfields, which have been largely treated as isolated in the community.

本文提出了一种名为RegBayes的新型计算框架，该框架执行正则化的贝叶斯后验推断，其灵活性优于通过先验调节专家知识的传统方法，并且涵盖面更广。RegBayes通过信息理论形式化之后，使正则化术语直接作用于期望的后验分布上，可以推断包括定向贝叶斯网络和无定向马尔科夫网络在内的各种模型，同时发掘潜在的预测特征，应用于分类和多任务学习。我们在多个基准数据集上进行的实证研究表明，RegBayes从大边界学习和贝叶斯非参数学习中继承了优点。

具有后验正则化的贝叶斯推断及其在无限潜在SVM中的应用