Image-based motion prediction is one of the essential techniques for robot
manipulation. Among the various prediction models, we focus on diffusion models
because they have achieved state-of-the-art performance in various
applications. In image-based motion prediction, diffusion models stochastically
predict contextually appropriate motion by gradually denoising random Gaussian
noise based on the image context. While diffusion models are able to predict
various motions by changing the random noise, they sometimes fail to predict a
contextually appropriate motion based on the image because the random noise is
sampled independently of the image context. To solve this problem, we propose
R2-Diff. In R2-Diff, a motion retrieved from a dataset based on image
similarity is fed into a diffusion model instead of random noise. Then, the
retrieved motion is refined through the denoising process of the diffusion
model. Since the retrieved motion is almost appropriate to the context, it
becomes easier to predict contextually appropriate motion. However, traditional
diffusion models are not optimized to refine the retrieved motion. Therefore,
we propose the method of tuning the hyperparameters based on the distance of
the nearest neighbor motion among the dataset to optimize the diffusion model
for refinement. Furthermore, we propose an image-based retrieval method to
retrieve the nearest neighbor motion in inference. Our proposed retrieval
efficiently computes the similarity based on the image features along the
motion trajectory. We demonstrate that R2-Diff accurately predicts appropriate
motions and achieves high task success rates compared to recent
state-of-the-art models in robot manipulation.

本文提出了一种名为 R2-Diff 的基于图像的运动预测方法，其中运动数据是从数据集中检索出的，通过调整模型超参数进行细化，并通过基于图像特征的检索方式来检索最近邻居运动。实验结果表明，R2-Diff 可以精确地预测适当的运动并在机器人操作中取得高任务成功率。