In recent years, modern techniques in deep learning and large-scale datasets
have led to impressive progress in 3D instance segmentation, grasp pose
estimation, and robotics. This allows for accurate detection directly in 3D
scenes, object- and environment-aware grasp prediction, as well as robust and
repeatable robotic manipulation. This work aims to integrate these recent
methods into a comprehensive framework for robotic interaction and manipulation
in human-centric environments. Specifically, we leverage 3D reconstructions
from a commodity 3D scanner for open-vocabulary instance segmentation,
alongside grasp pose estimation, to demonstrate dynamic picking of objects, and
opening of drawers. We show the performance and robustness of our model in two
sets of real-world experiments including dynamic object retrieval and drawer
opening, reporting a 51% and 82% success rate respectively. Code of our
framework as well as videos are available on: this https URL

将现代深度学习技术和大规模数据集应用于 3D 实例分割、抓取姿态估计和机器人学领域，以构建一个综合框架，实现在以人为中心的环境中的机器人交互和操控。

Spot-Compose: 点云中的开放词汇物体检索和抽屉操作框架

Spot-Compose: A Framework for Open-Vocabulary Object Retrieval and  Drawer Manipulation in Point Clouds

Anticipating the motion of all humans in dynamic environments such as homes
and offices is critical to enable safe and effective robot navigation. Such
spaces remain challenging as humans do not follow strict rules of motion and
there are often multiple occluded entry points such as corners and doors that
create opportunities for sudden encounters. In this work, we present a
Transformer based architecture to predict human future trajectories in
human-centric environments from input features including human positions, head
orientations, and 3D skeletal keypoints from onboard in-the-wild sensory
information. The resulting model captures the inherent uncertainty for future
human trajectory prediction and achieves state-of-the-art performance on common
prediction benchmarks and a human tracking dataset captured from a mobile robot
adapted for the prediction task. Furthermore, we identify new agents with
limited historical data as a major contributor to error and demonstrate the
complementary nature of 3D skeletal poses in reducing prediction error in such
challenging scenarios.

通过使用 Transformer 架构，从人体位置、头部朝向和三维关键点等输入特征来预测人类在人类中心环境中未来的轨迹，该模型捕捉了未来人类轨迹预测中的内在不确定性，并在常见预测基准和适用于预测任务的移动机器人跟踪数据集上取得了最先进的性能。在这种具有挑战性场景中，我们发现历史数据有限的新主体是错误的主要来源，并展示了三维骨骼姿势在减少预测误差方面的互补性。