This paper presents a novel optimization framework to address key challenges
presented by modern machine learning applications: High dimensionality,
distributional uncertainty, and data heterogeneity. Our approach unifies
regularized estimation, distributionally robust optimization (DRO), and
hierarchical Bayesian modeling in a single data-driven criterion. By employing
a hierarchical Dirichlet process (HDP) prior, the method effectively handles
multi-source data, achieving regularization, distributional robustness, and
borrowing strength across diverse yet related data-generating processes. We
demonstrate the method's advantages by establishing theoretical performance
guarantees and tractable Monte Carlo approximations based on Dirichlet process
(DP) theory. Numerical experiments validate the framework's efficacy in
improving and stabilizing both prediction and parameter estimation accuracy,
showcasing its potential for application in complex data environments.

该论文介绍了一种新颖的优化框架，用于解决现代机器学习应用中的关键挑战：高维度、分布不确定性和数据异质性。该方法通过使用分层狄利克雷过程（HDP）先验，有效处理多源数据，实现正则化、分布鲁棒性，并在不同但相关的数据生成过程之间共享强度。通过建立基于狄利克雷过程（DP）理论的理论性能保证和可处理的蒙特卡洛近似，我们展示了该方法的优势。数值实验证实了该框架在改善和稳定预测和参数估计精度方面的功效，展示了它在复杂数据环境中的应用潜力。

基于分层狄利克雷过程的分布鲁棒优化中的力量借助

Borrowing Strength in Distributionally Robust Optimization via  Hierarchical Dirichlet Processes

Bayesian quadrature optimization (BQO) maximizes the expectation of an
expensive black-box integrand taken over a known probability distribution. In
this work, we study BQO under distributional uncertainty in which the
underlying probability distribution is unknown except for a limited set of its
i.i.d. samples. A standard BQO approach maximizes the Monte Carlo estimate of
the true expected objective given the fixed sample set. Though Monte Carlo
estimate is unbiased, it has high variance given a small set of samples; thus
can result in a spurious objective function. We adopt the distributionally
robust optimization perspective to this problem by maximizing the expected
objective under the most adversarial distribution. In particular, we propose a
novel posterior sampling based algorithm, namely distributionally robust BQO
(DRBQO) for this purpose. We demonstrate the empirical effectiveness of our
proposed framework in synthetic and real-world problems, and characterize its
theoretical convergence via Bayesian regret.

本研究针对具有分布不确定性的贝叶斯积分优化问题使用分布鲁棒优化视角，提出了一种基于后验抽样的算法（DRBQO），旨在最大化最对抗分布下的预期目标，并通过贝叶斯遗憾度量其理论收敛性。我们在合成和真实世界问题中展示了我们提出的框架的实证有效性。

分布稳健的贝叶斯积分优化

Distributionally Robust Bayesian Quadrature Optimization

Estimating how uncertain an AI system is in its predictions is important to
improve the safety of such systems. Uncertainty in predictive can result from
uncertainty in model parameters, irreducible data uncertainty and uncertainty
due to distributional mismatch between the test and training data
distributions. Different actions might be taken depending on the source of the
uncertainty so it is important to be able to distinguish between them.
Recently, baseline tasks and metrics have been defined and several practical
methods to estimate uncertainty developed. These methods, however, attempt to
model uncertainty due to distributional mismatch either implicitly through
model uncertainty or as data uncertainty. This work proposes a new framework
for modeling predictive uncertainty called Prior Networks (PNs) which
explicitly models distributional uncertainty. PNs do this by parameterizing a
prior distribution over predictive distributions. This work focuses on
uncertainty for classification and evaluates PNs on the tasks of identifying
out-of-distribution (OOD) samples and detecting misclassification on the MNIST
dataset, where they are found to outperform previous methods. Experiments on
synthetic and MNIST and CIFAR-10 data show that unlike previous non-Bayesian
methods PNs are able to distinguish between data and distributional
uncertainty.

该研究提出了一种名为 Prior Networks（PNs）的框架来模型化分类任务中的数据和分布不确定性，并在 MNIST 数据集上实现了对 OOD 样本的识别和误分类的检测，与之前的方法相比，PNs 具有更好的表现。