Sim2real transfer is primarily concerned with transferring policies trained
in simulation to potentially noisy real world environments. A common problem
associated with sim2real transfer is estimating the real-world environmental
parameters to ground the simulated environment to. Although existing methods
such as Domain Randomisation (DR) can produce robust policies by sampling from
a distribution of parameters during training, there is no established method
for identifying the parameters of the corresponding distribution for a given
real-world setting. In this work, we propose Uncertainty-aware policy search
(UncAPS), where we use Universal Policy Network (UPN) to store
simulation-trained task-specific policies across the full range of
environmental parameters and then subsequently employ robust Bayesian
optimisation to craft robust policies for the given environment by combining
relevant UPN policies in a DR like fashion. Such policy-driven grounding is
expected to be more efficient as it estimates only task-relevant sets of
parameters. Further, we also account for the estimation uncertainties in the
search process to produce policies that are robust against both aleatoric and
epistemic uncertainties. We empirically evaluate our approach in a range of
noisy, continuous control environments, and show its improved performance
compared to competing baselines.

本文介绍了一种名为 Uncertainty-aware policy search 的策略搜索方法，通过结合与给定环境相关的 UPN 策略，采用类似 DR 的方法，使用鲁棒性贝叶斯优化来制定强鲁棒性策略。在一系列嘈杂的连续控制环境中的实验证明了该方法的有效性。