Excavators are crucial for diverse tasks such as construction and mining,
while autonomous excavator systems enhance safety and efficiency, address labor
shortages, and improve human working conditions. Different from the existing
modularized approaches, this paper introduces ExACT, an end-to-end autonomous
excavator system that processes raw LiDAR, camera data, and joint positions to
control excavator valves directly. Utilizing the Action Chunking with
Transformers (ACT) architecture, ExACT employs imitation learning to take
observations from multi-modal sensors as inputs and generate actionable
sequences. In our experiment, we build a simulator based on the captured
real-world data to model the relations between excavator valve states and joint
velocities. With a few human-operated demonstration data trajectories, ExACT
demonstrates the capability of completing different excavation tasks, including
reaching, digging and dumping through imitation learning in validations with
the simulator. To the best of our knowledge, ExACT represents the first
instance towards building an end-to-end autonomous excavator system via
imitation learning methods with a minimal set of human demonstrations. The
video about this work can be accessed at this https URL

通过模仿学习方法和最少一组人类演示，ExACT 是构建完全自主挖掘机系统的首个实例，其能够完成不同的挖掘任务。