Evolution Strategy (ES) algorithms have shown promising results in training complex robotic control policies due to their massive parallelism capability, simple implementation, effective parameter-space exploration, and fast training time. However, a key limitation of ES is its scalability to large capacity models, including modern neural network architectures. In this work, we develop Predictive Information Augmented Random Search (PI-ARS) to mitigate this limitation by leveraging recent advancements in representation learning to reduce the parameter search space for ES. Namely, PI-ARS combines a gradient-based representation learning technique, Predictive Information (PI), with a gradient-free ES algorithm, Augmented Random Search (ARS), to train policies that can process complex robot sensory inputs and handle highly nonlinear robot dynamics. We evaluate PI-ARS on a set of challenging visual-locomotion tasks where a quadruped robot needs to walk on uneven stepping stones, quincuncial piles, and moving platforms, as well as to complete an indoor navigation task. Across all tasks, PI-ARS demonstrates significantly better learning efficiency and performance compared to the ARS baseline. We further validate our algorithm by demonstrating that the learned policies can successfully transfer to a real quadruped robot, for example, achieving a 100% success rate on the real-world stepping stone environment, dramatically improving prior results achieving 40% success.

本研究中，我们开发了“预测信息增强随机搜索”（PI-ARS）方法，它结合了基于梯度的表示学习技术（PI）和基于梯度的无模型演化策略算法(ARS)，以训练能够处理复杂机器人感知输入和处理高度非线性机器人动态的策略。该方法在足够复杂且难以处理的视觉驱动机器人任务中得到验证，相比于ARS的基线算法，在学习效率和性能方面均有显著提升，实际应用中100％的成功率大幅提高了之前40％的效果。

PI-ARS：用预测信息表现加速演化学得的视觉 - 运动