Human beings construct perception of space by integrating sparse observations
into massively interconnected synapses and neurons, offering a superior
parallelism and efficiency. Replicating this capability in AI finds wide
applications in medical imaging, AR/VR, and embodied AI, where input data is
often sparse and computing resources are limited. However, traditional signal
reconstruction methods on digital computers face both software and hardware
challenges. On the software front, difficulties arise from storage
inefficiencies in conventional explicit signal representation. Hardware
obstacles include the von Neumann bottleneck, which limits data transfer
between the CPU and memory, and the limitations of CMOS circuits in supporting
parallel processing. We propose a systematic approach with software-hardware
co-optimizations for signal reconstruction from sparse inputs. Software-wise,
we employ neural field to implicitly represent signals via neural networks,
which is further compressed using low-rank decomposition and structured
pruning. Hardware-wise, we design a resistive memory-based computing-in-memory
(CIM) platform, featuring a Gaussian Encoder (GE) and an MLP Processing Engine
(PE). The GE harnesses the intrinsic stochasticity of resistive memory for
efficient input encoding, while the PE achieves precise weight mapping through
a Hardware-Aware Quantization (HAQ) circuit. We demonstrate the system's
efficacy on a 40nm 256Kb resistive memory-based in-memory computing macro,
achieving huge energy efficiency and parallelism improvements without
compromising reconstruction quality in tasks like 3D CT sparse reconstruction,
novel view synthesis, and novel view synthesis for dynamic scenes. This work
advances the AI-driven signal restoration technology and paves the way for
future efficient and robust medical AI and 3D vision applications.

我们提出了一种软硬件协同优化的系统方法，用于从稀疏输入中进行信号重建，软件层面上使用神经场通过神经网络隐式表示信号，并使用低秩分解和结构化剪枝进一步压缩；硬件层面上，设计了一种基于可变电阻存储器的计算内存（CIM）平台，其中包括高斯编码器（GE）和多层感知器处理引擎（PE）。此工作推进了以人工智能为驱动的信号恢复技术，并为未来高效稳健的医疗人工智能和三维视觉应用铺平了道路。