Deep generative models have various content creation applications such as
graphic design, e-commerce, and virtual Try-on. However, current works mainly
focus on synthesizing realistic visual outputs, often ignoring other sensory
modalities, such as touch, which limits physical interaction with users. In
this work, we leverage deep generative models to create a multi-sensory
experience where users can touch and see the synthesized object when sliding
their fingers on a haptic surface. The main challenges lie in the significant
scale discrepancy between vision and touch sensing and the lack of explicit
mapping from touch sensing data to a haptic rendering device. To bridge this
gap, we collect high-resolution tactile data with a GelSight sensor and create
a new visuotactile clothing dataset. We then develop a conditional generative
model that synthesizes both visual and tactile outputs from a single sketch. We
evaluate our method regarding image quality and tactile rendering accuracy.
Finally, we introduce a pipeline to render high-quality visual and tactile
outputs on an electroadhesion-based haptic device for an immersive experience,
allowing for challenging materials and editable sketch inputs.

本研究使用深度生成模型创建一个多感官体验，用户可以在触觉表面上滑动手指，触摸和查看合成物体，并提出了一种新的视触觉服装数据集和条件生成模型以合成视觉和触觉输出，并介绍了一个流水线，在基于电除尘的触觉设备上呈现高质量的视觉和触觉输出，实现沉浸式体验。

可控视觉触觉合成

Controllable Visual-Tactile Synthesis

This article presents a novel telepresence system for advancing aerial
manipulation in dynamic and unstructured environments. The proposed system not
only features a haptic device, but also a virtual reality (VR) interface that
provides real-time 3D displays of the robot's workspace as well as a haptic
guidance to its remotely located operator. To realize this, multiple sensors
namely a LiDAR, cameras and IMUs are utilized. For processing of the acquired
sensory data, pose estimation pipelines are devised for industrial objects of
both known and unknown geometries. We further propose an active learning
pipeline in order to increase the sample efficiency of a pipeline component
that relies on Deep Neural Networks (DNNs) based object detection. All these
algorithms jointly address various challenges encountered during the execution
of perception tasks in industrial scenarios. In the experiments, exhaustive
ablation studies are provided to validate the proposed pipelines.
Methodologically, these results commonly suggest how an awareness of the
algorithms' own failures and uncertainty (`introspection') can be used tackle
the encountered problems. Moreover, outdoor experiments are conducted to
evaluate the effectiveness of the overall system in enhancing aerial
manipulation capabilities. In particular, with flight campaigns over days and
nights, from spring to winter, and with different users and locations, we
demonstrate over 70 robust executions of pick-and-place, force application and
peg-in-hole tasks with the DLR cable-Suspended Aerial Manipulator (SAM). As a
result, we show the viability of the proposed system in future industrial
applications.

本文提出了一种新型遥控系统，包含了触觉设备以及虚拟现实界面，应用于动态和非结构化环境下的空中机器人操作。利用多个传感器进行感知、姿态估计以及神经网络的主动学习等算法，针对工业领域的任务进行了验证实验，并展示了该系统在未来工业应用中的可行性。

基于物体姿态估计和主动学习的虚拟现实：实现具有空中操纵能力的遥控机器人

Virtual Reality via Object Pose Estimation and Active Learning: Realizing Telepresence Robots with Aerial Manipulation Capabilities

This paper presents a novel telepresence system for enhancing aerial
manipulation capabilities. It involves not only a haptic device, but also a
virtual reality that provides a 3D visual feedback to a remotely-located
teleoperator in real-time. We achieve this by utilizing onboard visual and
inertial sensors, an object tracking algorithm and a pre-generated object
database. As the virtual reality has to closely match the real remote scene, we
propose an extension of a marker tracking algorithm with visual-inertial
odometry. Both indoor and outdoor experiments show benefits of our proposed
system in achieving advanced aerial manipulation tasks, namely grasping,
placing, force exertion and peg-in-hole insertion.

本文提出了一种新型的遥操作系统，其结合了触觉设备和虚拟现实，为遥操作者提供 3D 视觉反馈，通过利用机载视觉和惯性传感器、对象跟踪算法和对象数据库实现。通过室内和室外实验，我们证明了提出的系统在实现高级空中操作任务方面的优势，包括抓取、放置、施力和插销孔插入。