Goal-conditioned rearrangement of deformable objects (e.g. straightening a
rope and folding a cloth) is one of the most common deformable manipulation
tasks, where the robot needs to rearrange a deformable object into a prescribed
goal configuration with only visual observations. These tasks are typically
confronted with two main challenges: the high dimensionality of deformable
configuration space and the underlying complexity, nonlinearity and uncertainty
inherent in deformable dynamics. To address these challenges, we propose a
novel representation strategy that can efficiently model the deformable object
states with a set of keypoints and their interactions. We further propose
local-graph neural network (GNN), a light local GNN learning to jointly model
the deformable rearrangement dynamics and infer the optimal manipulation
actions (e.g. pick and place) by constructing and updating two dynamic graphs.
Both simulated and real experiments have been conducted to demonstrate that the
proposed dynamic graph representation shows superior expressiveness in modeling
deformable rearrangement dynamics. Our method reaches much higher success rates
on a variety of deformable rearrangement tasks (96.3% on average) than
state-of-the-art method in simulation experiments. Besides, our method is much
more lighter and has a 60% shorter inference time than state-of-the-art
methods. We also demonstrate that our method performs well in the multi-task
learning scenario and can be transferred to real-world applications with an
average success rate of 95% by solely fine tuning a keypoint detector.

通过建立 keypoints 和它们之间的相互作用的一组动态图表达的新颖表示策略，提出一种在视觉观测的情况下模拟可变形对象重新排列动态并推断最佳操纵动作的方法。新方法在模拟实验中显示出优越的表达能力，在各种可变形重新排列任务中的成功率远高于现有方法（平均 96.3%），同时更轻量且推理时间较现有方法短 60%。并且该方法在多任务学习场景中表现良好，并通过仅微调关键点检测器可以转移到实际应用中，平均成功率为 95%。