In this letter, we introduce ViHOPE, a novel framework for estimating the 6D
pose of an in-hand object using visuotactile perception. Our key insight is
that the accuracy of the 6D object pose estimate can be improved by explicitly
completing the shape of the object. To this end, we introduce a novel
visuotactile shape completion module that uses a conditional Generative
Adversarial Network to complete the shape of an in-hand object based on
volumetric representation. This approach improves over prior works that
directly regress visuotactile observations to a 6D pose. By explicitly
completing the shape of the in-hand object and jointly optimizing the shape
completion and pose estimation tasks, we improve the accuracy of the 6D object
pose estimate. We train and test our model on a synthetic dataset and compare
it with the state-of-the-art. In the visuotactile shape completion task, we
outperform the state-of-the-art by 265% using the Intersection of Union metric
and achieve 88% lower Chamfer Distance. In the visuotactile pose estimation
task, we present results that suggest our framework reduces position and
angular errors by 35% and 64%, respectively. Furthermore, we ablate our
framework to confirm the gain on the 6D object pose estimate from explicitly
completing the shape. Ultimately, we show that our framework produces models
that are robust to sim-to-real transfer on a real-world robot platform.

提出了一种新颖的 ViHOPE 框架，用于利用视触知感知估计手持物体的六维位姿，通过显式完成物体的形状，提高了六维物体位姿估计的准确性。

ViHOPE：视触觉手持物体 6D 姿态估计与形状补全

ViHOPE: Visuotactile In-Hand Object 6D Pose Estimation with Shape  Completion

Existing robotic systems have a clear tension between generality and
precision. Deployed solutions for robotic manipulation tend to fall into the
paradigm of one robot solving a single task, lacking precise generalization,
i.e., the ability to solve many tasks without compromising on precision. This
paper explores solutions for precise and general pick-and-place. In precise
pick-and-place, i.e. kitting, the robot transforms an unstructured arrangement
of objects into an organized arrangement, which can facilitate further
manipulation. We propose simPLE (simulation to Pick Localize and PLacE) as a
solution to precise pick-and-place. simPLE learns to pick, regrasp and place
objects precisely, given only the object CAD model and no prior experience. We
develop three main components: task-aware grasping, visuotactile perception,
and regrasp planning. Task-aware grasping computes affordances of grasps that
are stable, observable, and favorable to placing. The visuotactile perception
model relies on matching real observations against a set of simulated ones
through supervised learning. Finally, we compute the desired robot motion by
solving a shortest path problem on a graph of hand-to-hand regrasps. On a
dual-arm robot equipped with visuotactile sensing, we demonstrate
pick-and-place of 15 diverse objects with simPLE. The objects span a wide range
of shapes and simPLE achieves successful placements into structured
arrangements with 1mm clearance over 90% of the time for 6 objects, and over
80% of the time for 11 objects. Videos are available at
this http URL .

传统机器人系统在通用性与精确性之间存在明显的紧张关系。本文探讨了精确和通用的拾取和放置解决方案，基于 simPLE（模拟选择定位和放置）的方法，通过任务感知抓取、视触感知和再抓计划的三个主要组件，成功实现了对 15 个不同形状对象的精确拾取放置。