Visual sensors, including 3D LiDAR, neuromorphic DVS sensors, and
conventional frame cameras, are increasingly integrated into edge-side
intelligent machines. Realizing intensive multi-sensory data analysis directly
on edge intelligent machines is crucial for numerous emerging edge
applications, such as augmented and virtual reality and unmanned aerial
vehicles, which necessitates unified data representation, unprecedented
hardware energy efficiency and rapid model training. However, multi-sensory
data are intrinsically heterogeneous, causing significant complexity in the
system development for edge-side intelligent machines. In addition, the
performance of conventional digital hardware is limited by the physically
separated processing and memory units, known as the von Neumann bottleneck, and
the physical limit of transistor scaling, which contributes to the slowdown of
Moore's law. These limitations are further intensified by the tedious training
of models with ever-increasing sizes. We propose a novel hardware-software
co-design, random resistive memory-based deep extreme point learning machine
(DEPLM), that offers efficient unified point set analysis. We show the system's
versatility across various data modalities and two different learning tasks.
Compared to a conventional digital hardware-based system, our co-design system
achieves huge energy efficiency improvements and training cost reduction when
compared to conventional systems. Our random resistive memory-based deep
extreme point learning machine may pave the way for energy-efficient and
training-friendly edge AI across various data modalities and tasks.

通过硬件软件的协同设计，我们提出了基于随机电阻存储器的深度极端点学习机，实现了高效统一的点集分析，比传统系统节省了大量能源并降低了训练成本，为各种数据模态和任务提供了节能高效的边缘人工智能解决方案。