Precise manipulation that is generalizable across scenes and objects remains
a persistent challenge in robotics. Current approaches for this task heavily
depend on having a significant number of training instances to handle objects
with pronounced visual and/or geometric part ambiguities. Our work explores the
grounding of fine-grained part descriptors for precise manipulation in a
zero-shot setting by utilizing web-trained text-to-image diffusion-based
generative models. We tackle the problem by framing it as a dense semantic part
correspondence task. Our model returns a gripper pose for manipulating a
specific part, using as reference a user-defined click from a source image of a
visually different instance of the same object. We require no manual grasping
demonstrations as we leverage the intrinsic object geometry and features.
Practical experiments in a real-world tabletop scenario validate the efficacy
of our approach, demonstrating its potential for advancing semantic-aware
robotics manipulation. Web page: this https URL

利用网络训练的文本到图像扩散生成模型，在无样本情况下对细粒度部件描述符进行准确操作，通过将问题框架化为密集语义部件对应任务，返回用于操作特定部件的夹爪位姿，无需手动示教，验证了该方法在真实世界的桌面场景中的实验，证明了其推进语义感知机器人操作的潜力。

点击抓取：通过视觉扩散描述符实现零射击精确操控

Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion  Descriptors

Benchmarks for robot manipulation are crucial to measuring progress in the
field, yet there are few benchmarks that demonstrate critical manipulation
skills, possess standardized metrics, and can be attempted by a wide array of
robot platforms. To address a lack of such benchmarks, we propose Rubik's cube
manipulation as a benchmark to measure simultaneous performance of precise
manipulation and sequential manipulation. The sub-structure of the Rubik's cube
demands precise positioning of the robot's end effectors, while its highly
reconfigurable nature enables tasks that require the robot to manage pose
uncertainty throughout long sequences of actions. We present a protocol for
quantitatively measuring both the accuracy and speed of Rubik's cube
manipulation. This protocol can be attempted by any general-purpose
manipulator, and only requires a standard 3x3 Rubik's cube and a flat surface
upon which the Rubik's cube initially rests (e.g. a table). We demonstrate this
protocol for two distinct baseline approaches on a PR2 robot. The first
baseline provides a fundamental approach for pose-based Rubik's cube
manipulation. The second baseline demonstrates the benchmark's ability to
quantify improved performance by the system, particularly that resulting from
the integration of pre-touch sensing. To demonstrate the benchmark's
applicability to other robot platforms and algorithmic approaches, we present
the functional blocks required to enable the HERB robot to manipulate the
Rubik's cube via push-grasping.

提出 “魔方操纵” 作为机器人操纵领域的基准测试，以衡量精准操纵和顺序操纵的同时表现，量化精度和速度，并演示该协议在 PR2 和 HERB 机器人上的基础方法和改进方法