Clothes grasping and unfolding is a core step in robotic-assisted dressing.
Most existing works leverage depth images of clothes to train a deep
learning-based model to recognize suitable grasping points. These methods often
utilize physics engines to synthesize depth images to reduce the cost of real
labeled data collection. However, the natural domain gap between synthetic and
real images often leads to poor performance of these methods on real data.
Furthermore, these approaches often struggle in scenarios where grasping points
are occluded by the clothing item itself. To address the above challenges, we
propose a novel Bi-directional Fractal Cross Fusion Network (BiFCNet) for
semantic segmentation, enabling recognition of graspable regions in order to
provide more possibilities for grasping. Instead of using depth images only, we
also utilize RGB images with rich color features as input to our network in
which the Fractal Cross Fusion (FCF) module fuses RGB and depth data by
considering global complex features based on fractal geometry. To reduce the
cost of real data collection, we further propose a data augmentation method
based on an adversarial strategy, in which the color and geometric
transformations simultaneously process RGB and depth data while maintaining the
label correspondence. Finally, we present a pipeline for clothes grasping and
unfolding from the perspective of semantic segmentation, through the addition
of a strategy for grasp point selection from segmentation regions based on
clothing flatness measures, while taking into account the grasping direction.
We evaluate our BiFCNet on the public dataset NYUDv2 and obtained comparable
performance to current state-of-the-art models. We also deploy our model on a
Baxter robot, running extensive grasping and unfolding experiments as part of
our ablation studies, achieving an 84% success rate.

本文提出了一种新颖的双向分形交叉融合神经网络（BiFCNet）用于语义分割，使得机器人能够识别出可抓取的区域以提供更多抓取可能性，同时还提出了一种基于对抗方法的数据增强方法，最后基于衣服平整度量的分割区域抓取点选择策略。在公共数据集 NYUDv2 上评估这种方法，取得与目前最先进模型相当的表现，并且通过对机械臂的抓取和展开实验进行了验证，获得了 84% 的成功率。