In order to successfully perform manipulation tasks in new environments, such
as grasping, robots must be proficient in segmenting unseen objects from the
background and/or other objects. Previous works perform unseen object instance
segmentation (UOIS) by training deep neural networks on large-scale data to
learn RGB/RGB-D feature embeddings, where cluttered environments often result
in inaccurate segmentations. We build upon these methods and introduce a novel
approach to correct inaccurate segmentation, such as under-segmentation, of
static image-based UOIS masks by using robot interaction and a designed body
frame-invariant feature. We demonstrate that the relative linear and rotational
velocities of frames randomly attached to rigid bodies due to robot
interactions can be used to identify objects and accumulate corrected
object-level segmentation masks. By introducing motion to regions of
segmentation uncertainty, we are able to drastically improve segmentation
accuracy in an uncertainty-driven manner with minimal, non-disruptive
interactions (ca. 2-3 per scene). We demonstrate the effectiveness of our
proposed interactive perception pipeline in accurately segmenting cluttered
scenes by achieving an average object segmentation accuracy rate of 80.7%, an
increase of 28.2% when compared with other state-of-the-art UOIS methods.

通过使用机器人的互动和设计的具有变换不变性的体框特征，我们提出了一种改正不准确分割的新方法，可以显著提高不确定性驱动的分割准确性，并实现平均 80.7% 的物体分割准确率，相较其他最先进的无人物体实例分割方法提高了 28.2%。

RISeg：机器人通过身体帧不变特征进行交互式目标分割

RISeg: Robot Interactive Object Segmentation via Body Frame-Invariant  Features

Segmenting unseen objects in cluttered scenes is an important skill that
robots need to acquire in order to perform tasks in new environments. In this
work, we propose a new method for unseen object instance segmentation by
learning RGB-D feature embeddings from synthetic data. A metric learning loss
function is utilized to learn to produce pixel-wise feature embeddings such
that pixels from the same object are close to each other and pixels from
different objects are separated in the embedding space. With the learned
feature embeddings, a mean shift clustering algorithm can be applied to
discover and segment unseen objects. We further improve the segmentation
accuracy with a new two-stage clustering algorithm. Our method demonstrates
that non-photorealistic synthetic RGB and depth images can be used to learn
feature embeddings that transfer well to real-world images for unseen object
instance segmentation.

通过学习来自合成数据的 RGB-D 特征嵌入，引入了一种新的方法来实现未知物体实例的分割，其中使用度量学习损失函数来产生像素级特征嵌入，以便在嵌入空间中将同一对象的像素靠近，并将不同对象的像素分开。通过使用学习到的特征嵌入，可以将平均漂移聚类算法应用于发现和分段未知对象，对此进一步改进了分段的准确性。我们的方法证明，非真实光线的合成 RGB 和深度图像可以用于学习特征嵌入，从而实现在未知物体实例分割方面的好的迁移效果。

学习 RGB-D 特征嵌入以进行未见过的物体实例分割

Learning RGB-D Feature Embeddings for Unseen Object Instance  Segmentation

In order to function in unstructured environments, robots need the ability to
recognize unseen novel objects. We take a step in this direction by tackling
the problem of segmenting unseen object instances in tabletop environments.
However, the type of large-scale real-world dataset required for this task
typically does not exist for most robotic settings, which motivates the use of
synthetic data. We propose a novel method that separately leverages synthetic
RGB and synthetic depth for unseen object instance segmentation. Our method is
comprised of two stages where the first stage operates only on depth to produce
rough initial masks, and the second stage refines these masks with RGB.
Surprisingly, our framework is able to learn from synthetic RGB-D data where
the RGB is non-photorealistic. To train our method, we introduce a large-scale
synthetic dataset of random objects on tabletops. We show that our method,
trained on this dataset, can produce sharp and accurate masks, outperforming
state-of-the-art methods on unseen object instance segmentation. We also show
that our method can segment unseen objects for robot grasping. Code, models and
video can be found at
this https URL

本篇论文提出了一种使用合成 RGB 和深度图像数据实现表面外目标实例分割的新方法，该方法能够学习非真实感 RGB-D 数据，并在机器人抓取任务中表现出色。