Controlling hand exoskeletons to assist individuals with grasping tasks poses
a challenge due to the difficulty in understanding user intentions. We propose
that most daily grasping tasks during activities of daily living (ADL) can be
deduced by analyzing object geometries (simple and complex) from 3D point
clouds. The study introduces PointGrasp, a real-time system designed for
identifying household scenes semantically, aiming to support and enhance
assistance during ADL for tailored end-to-end grasping tasks. The system
comprises an RGB-D camera with an inertial measurement unit and a
microprocessor integrated into a tendon-driven soft robotic glove. The RGB-D
camera processes 3D scenes at a rate exceeding 30 frames per second. The
proposed pipeline demonstrates an average RMSE of 0.8 $\pm$ 0.39 cm for simple
and 0.11 $\pm$ 0.06 cm for complex geometries. Within each mode, it identifies
and pinpoints reachable objects. This system shows promise in end-to-end
vision-driven robotic-assisted rehabilitation manual tasks.

通过分析三维点云中的对象几何形状（简单和复杂），提出了一种基于 PointGrasp 的实时系统，用于识别日常生活中的家居场景，并支持和增强日常活动中的定制化端到端抓取任务，实现可达对象的定位和识别。该系统在简单和复杂几何形状的情况下，平均 RMSE 为 0.8±0.39 cm 和 0.11±0.06 cm，展示了在视觉驱动的机器人辅助康复手动任务中的潜力。

点云感知：基于点云的张力驱动软机器人手套应用中的抓取

PointGrasp: Point Cloud-based Grasping for Tendon-driven Soft Robotic  Glove Applications

Well structured visual representations can make robot learning faster and can
improve generalization. In this paper, we study how we can acquire effective
object-centric representations for robotic manipulation tasks without human
labeling by using autonomous robot interaction with the environment. Such
representation learning methods can benefit from continuous refinement of the
representation as the robot collects more experience, allowing them to scale
effectively without human intervention. Our representation learning approach is
based on object persistence: when a robot removes an object from a scene, the
representation of that scene should change according to the features of the
object that was removed. We formulate an arithmetic relationship between
feature vectors from this observation, and use it to learn a representation of
scenes and objects that can then be used to identify object instances, localize
them in the scene, and perform goal-directed grasping tasks where the robot
must retrieve commanded objects from a bin. The same grasping procedure can
also be used to automatically collect training data for our method, by
recording images of scenes, grasping and removing an object, and recording the
outcome. Our experiments demonstrate that this self-supervised approach for
tasked grasping substantially outperforms direct reinforcement learning from
images and prior representation learning methods.

本文探讨了如何通过自主机器人与环境进行交互，获得适用于机器人操纵任务的有效物体中心表示，无需人为标注。基于对象持久性的表示学习方法能够随着机器人收集更多经验，不断改进表示，从而可以在无需人为干预的情况下扩展规模。我们的实验表明，这种自我监督方法显着优于直接从图像进行强化学习和先前的表示学习方法。