Fine manipulation tasks, such as threading cable ties or slotting a battery,
are notoriously difficult for robots because they require precision, careful
coordination of contact forces, and closed-loop visual feedback. Performing
these tasks typically requires high-end robots, accurate sensors, or careful
calibration, which can be expensive and difficult to set up. Can learning
enable low-cost and imprecise hardware to perform these fine manipulation
tasks? We present a low-cost system that performs end-to-end imitation learning
directly from real demonstrations, collected with a custom teleoperation
interface. Imitation learning, however, presents its own challenges,
particularly in high-precision domains: errors in the policy can compound over
time, and human demonstrations can be non-stationary. To address these
challenges, we develop a simple yet novel algorithm, Action Chunking with
Transformers (ACT), which learns a generative model over action sequences. ACT
allows the robot to learn 6 difficult tasks in the real world, such as opening
a translucent condiment cup and slotting a battery with 80-90% success, with
only 10 minutes worth of demonstrations. Project website:
this https URL

我们提出了一种低成本的系统，该系统可以直接从真实演示中进行端到端模仿学习，以学习在现实世界中执行精细操作任务的方法。

使用低成本硬件学习精细双手操作

Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

Vision-based tactile sensors have the potential to provide important contact
geometry to localize the objective with visual occlusion. However, it is
challenging to measure high-resolution 3D contact geometry for a compact robot
finger, to simultaneously meet optical and mechanical constraints. In this
work, we present the GelSight Wedge sensor, which is optimized to have a
compact shape for robot fingers, while achieving high-resolution 3D
reconstruction. We evaluate the 3D reconstruction under different lighting
configurations, and extend the method from 3 lights to 1 or 2 lights. We
demonstrate the flexibility of the design by shrinking the sensor to the size
of a human finger for fine manipulation tasks. We also show the effectiveness
and potential of the reconstructed 3D geometry for pose tracking in the 3D
space.

本文介绍了一种经过优化的 GelSight 楔形传感器，其具有紧凑的形状用于机器人手指，并同时满足光学和机械约束，可高精度重建 3D 接触几何。作者在不同光照条件下评估了该传感器的 3D 重建，最终将传感器缩小到人手指的尺寸，演示了 3D 几何重建的效果和潜力，同时证明了该传感器对于 3D 空间中的姿态跟踪以及微小操纵的适用性。

GelSight 楔形：使用紧凑机器人手指测量高分辨率的 3D 接触几何形状

GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a  Compact Robot Finger

With the advent of Generative Adversarial Networks (GANs), a finer level of
control in manipulating various features of an image has become possible. One
example of such fine manipulation is changing the position of the light source
in a scene. This is fundamentally an ill-posed problem, since it requires
understanding the scene geometry to generate proper lighting effects. This
problem is not a trivial one and can become even more complicated if we want to
change the direction of the light source from any direction to a specific one.
Here we provide our attempt to solve this problem using GANs. Specifically,
pix2pix [arXiv:1611.07004] trained with the dataset VIDIT [arXiv:2005.05460]
which contains images of the same scene with different types of light
temperature and 8 different light source positions (N, NE, E, SE, S, SW, W,
NW). The results are 8 neural networks trained to be able to change the
direction of the light source from any direction to one of the 8 previously
mentioned. Additionally, we provide, as a tool, a simple CNN trained to
identify the direction of the light source in an image.

使用 pix2pix 训练网络解决了改变场景中光源方向的问题，该网络通过生成对抗神经网络和 VIDIT 数据集训练，可以将任意方向的光源转换为 8 个预先指定的方向，并提供了一个用于识别图像中光源方向的简单 CNN 工具。