In this paper, we propose the use of generative artificial intelligence (AI)
to improve zero-shot performance of a pre-trained policy by altering
observations during inference. Modern robotic systems, powered by advanced
neural networks, have demonstrated remarkable capabilities on pre-trained
tasks. However, generalizing and adapting to new objects and environments is
challenging, and fine-tuning visuomotor policies is time-consuming. To overcome
these issues we propose Robotic Policy Inference via Synthetic Observations
(ROSO). ROSO uses stable diffusion to pre-process a robot's observation of
novel objects during inference time to fit within its distribution of
observations of the pre-trained policies. This novel paradigm allows us to
transfer learned knowledge from known tasks to previously unseen scenarios,
enhancing the robot's adaptability without requiring lengthy fine-tuning. Our
experiments show that incorporating generative AI into robotic inference
significantly improves successful outcomes, finishing up to 57% of tasks
otherwise unsuccessful with the pre-trained policy.

我们提出使用生成人工智能（AI）来改变推理过程中的观察，以提高预训练策略的零样本性能，并通过稳定的扩散来预处理机器人对新对象的观察，从而在未经漫长微调的情况下提高机器人的适应能力。