Home robots intend to make their users lives easier. Our work assists in this
goal by enabling robots to inform their users of dangerous or unsanitary
anomalies in their home. Some examples of these anomalies include the user
leaving their milk out, forgetting to turn off the stove, or leaving poison
accessible to children. To move towards enabling home robots with these
abilities, we have created a new dataset, which we call SafetyDetect. The
SafetyDetect dataset consists of 1000 anomalous home scenes, each of which
contains unsafe or unsanitary situations for an agent to detect. Our approach
utilizes large language models (LLMs) alongside both a graph representation of
the scene and the relationships between the objects in the scene. Our key
insight is that this connected scene graph and the object relationships it
encodes enables the LLM to better reason about the scene -- especially as it
relates to detecting dangerous or unsanitary situations. Our most promising
approach utilizes GPT-4 and pursues a categorization technique where object
relations from the scene graph are classified as normal, dangerous, unsanitary,
or dangerous for children. This method is able to correctly identify over 90%
of anomalous scenarios in the SafetyDetect Dataset. Additionally, we conduct
real world experiments on a ClearPath TurtleBot where we generate a scene graph
from visuals of the real world scene, and run our approach with no
modification. This setup resulted in little performance loss. The SafetyDetect
Dataset and code will be released to the public upon this papers publication.

通过利用大型语言模型和场景图，我们在安全检测数据集中使用 GPT-4 进行家庭机器人的安全检测，该方法能够正确识别超过 90％的异常情况。

饮用奶后请放回！用于使具备行动能力的智能体检测异常情况的数据集

"Don't forget to put the milk back!" Dataset for Enabling Embodied  Agents to Detect Anomalous Situations

Throughout history, we have successfully integrated various machines into our
homes. Dishwashers, laundry machines, stand mixers, and robot vacuums are a few
recent examples. However, these machines excel at performing only a single task
effectively. The concept of a "generalist machine" in homes - a domestic
assistant that can adapt and learn from our needs, all while remaining
cost-effective - has long been a goal in robotics that has been steadily
pursued for decades. In this work, we initiate a large-scale effort towards
this goal by introducing Dobb-E, an affordable yet versatile general-purpose
system for learning robotic manipulation within household settings. Dobb-E can
learn a new task with only five minutes of a user showing it how to do it,
thanks to a demonstration collection tool ("The Stick") we built out of cheap
parts and iPhones. We use the Stick to collect 13 hours of data in 22 homes of
New York City, and train Home Pretrained Representations (HPR). Then, in a
novel home environment, with five minutes of demonstrations and fifteen minutes
of adapting the HPR model, we show that Dobb-E can reliably solve the task on
the Stretch, a mobile robot readily available on the market. Across roughly 30
days of experimentation in homes of New York City and surrounding areas, we
test our system in 10 homes, with a total of 109 tasks in different
environments, and finally achieve a success rate of 81%. Beyond success
percentages, our experiments reveal a plethora of unique challenges absent or
ignored in lab robotics. These range from effects of strong shadows, to
variable demonstration quality by non-expert users. With the hope of
accelerating research on home robots, and eventually seeing robot butlers in
every home, we open-source Dobb-E software stack and models, our data, and our
hardware designs at this https URL

通过引入具有成本效益的多功能家庭机器人系统 Dobb-E，该研究开启了对家庭机器人学习机器人操作的大规模研究，并在各种环境中进行了实验，取得了 81% 的成功率。

关于将机器人引入家庭的研究

On Bringing Robots Home

Vision systems mounted on home robots need to interact with unseen classes in
changing environments. Robots have limited computational resources, labelled
data and storage capability. These requirements pose some unique challenges:
models should adapt without forgetting past knowledge in a data- and
parameter-efficient way. We characterize the problem as few-shot (FS) online
continual learning (OCL), where robotic agents learn from a non-repeated stream
of few-shot data updating only a few model parameters. Additionally, such
models experience variable conditions at test time, where objects may appear in
different poses (e.g., horizontal or vertical) and environments (e.g., day or
night). To improve robustness of CL agents, we propose RobOCLe, which; 1)
constructs an enriched feature space computing high order statistical moments
from the embedded features of samples; and 2) computes similarity between high
order statistics of the samples on the enriched feature space, and predicts
their class labels. We evaluate robustness of CL models to train/test
augmentations in various cases. We show that different moments allow RobOCLe to
capture different properties of deformations, providing higher robustness with
no decrease of inference speed.

家用机器人上安装的视觉系统需要与不可见的类别在多变的环境中进行交互。我们提出了 RobOCLe 作为一种少样本在线持续学习模型，通过构建富化特征空间和计算样本的高阶统计矩来改善连续学习模型的鲁棒性。

在线持续学习用于室内物体识别的强健性

Online Continual Learning for Robust Indoor Object Recognition

Every home is different, and every person likes things done in their
particular way. Therefore, home robots of the future need to both reason about
the sequential nature of day-to-day tasks and generalize to user's preferences.
To this end, we propose a Transformer Task Planner(TTP) that learns high-level
actions from demonstrations by leveraging object attribute-based
representations. TTP can be pre-trained on multiple preferences and shows
generalization to unseen preferences using a single demonstration as a prompt
in a simulated dishwasher loading task. Further, we demonstrate real-world dish
rearrangement using TTP with a Franka Panda robotic arm, prompted using a
single human demonstration.

该文章提出了一种建立在物体属性表示基础上的 Transformer 任务规划器 (TTP)，可从多个偏好中进行预训练，并可以使用单个演示作为提示在模拟洗碗机任务中展现出对未见过偏好的泛化能力，在实际任务中使用 Franka Panda 机械臂完成了盘子重新摆放任务。