In this paper, we tackle the problem of pushing piles of small objects into a
desired target set using visual feedback. Unlike conventional single-object
manipulation pipelines, which estimate the state of the system parametrized by
pose, the underlying physical state of this system is difficult to observe from
images. Thus, we take the approach of reasoning directly in the space of
images, and acquire the dynamics of visual measurements in order to synthesize
a visual-feedback policy. We present a simple controller using an image-space
Lyapunov function, and evaluate the closed-loop performance using three
different class of models for image prediction: deep-learning-based models for
image-to-image translation, an object-centric model obtained from treating each
pixel as a particle, and a switched-linear system where an action-dependent
linear map is used. Through results in simulation and experiment, we show that
for this task, a linear model works surprisingly well -- achieving better
prediction error, downstream task performance, and generalization to new
environments than the deep models we trained on the same amount of data. We
believe these results provide an interesting example in the spectrum of models
that are most useful for vision-based feedback in manipulation, considering
both the quality of visual prediction, as well as compatibility with rigorous
methods for control design and analysis. Project site:
this https URL

本文研究通过视觉反馈将小物品推到目标点的问题，提出了一个基于图像空间的李亚普诺夫函数的简单控制器，评估了三种不同类别的图像预测模型在闭环控制中的性能，得到线性模型在预测错误、下游任务性能和泛化性能方面比深度模型更好的结果。