Physical neural networks (PNNs) are a class of neural-like networks that
leverage the properties of physical systems to perform computation. While PNNs
are so far a niche research area with small-scale laboratory demonstrations,
they are arguably one of the most underappreciated important opportunities in
modern AI. Could we train AI models 1000x larger than current ones? Could we do
this and also have them perform inference locally and privately on edge
devices, such as smartphones or sensors? Research over the past few years has
shown that the answer to all these questions is likely "yes, with enough
research": PNNs could one day radically change what is possible and practical
for AI systems. To do this will however require rethinking both how AI models
work, and how they are trained - primarily by considering the problems through
the constraints of the underlying hardware physics. To train PNNs at large
scale, many methods including backpropagation-based and backpropagation-free
approaches are now being explored. These methods have various trade-offs, and
so far no method has been shown to scale to the same scale and performance as
the backpropagation algorithm widely used in deep learning today. However, this
is rapidly changing, and a diverse ecosystem of training techniques provides
clues for how PNNs may one day be utilized to create both more efficient
realizations of current-scale AI models, and to enable unprecedented-scale
models.

物理神经网络（PNN）是一种利用物理系统特性进行计算的类神经网络，其有潜力在现代人工智能领域引发革新，尤其在训练大规模的 AI 模型以及探索新的训练技术方面尤为重要。

物理神经网络训练

Training of Physical Neural Networks

Physical neural networks (PNNs) are emerging paradigms for neural network
acceleration due to their high-bandwidth, in-propagation analogue processing.
Despite the advantages of PNN for inference, training remains a challenge. The
imperfect information of the physical transformation means the failure of
conventional gradient-based updates from backpropagation (BP). Here, we present
the asymmetrical training (AT) method, which treats the PNN structure as a grey
box. AT performs training while only knowing the last layer output and neuron
topological connectivity of a deep neural network structure, not requiring
information about the physical control-transformation mapping. We
experimentally demonstrated the AT method on deep grey-box PNNs implemented by
uncalibrated photonic integrated circuits (PICs), improving the classification
accuracy of Iris flower and modified MNIST hand-written digits from random
guessing to near theoretical maximum. We also showcased the consistently
enhanced performance of AT over BP for different datasets, including MNIST,
fashion-MNIST, and Kuzushiji-MNIST. The AT method demonstrated successful
training with minimal hardware overhead and reduced computational overhead,
serving as a robust light-weight training alternative to fully explore the
advantages of physical computation.

物理神经网络（PNN）作为神经网络加速的新范式，由于其高带宽、传播中的模拟处理等特点而备受关注。然而，尽管 PNN 在推理方面具有优势，但训练仍然面临挑战。为了克服这个问题，我们提出了一种称为 “不对称训练（AT）” 的方法，该方法将 PNN 结构作为一个灰盒子进行处理。通过实验证明，在未校准的光子集成电路（PIC）实现的深度灰盒 PNN 上使用 AT 方法能够将 Iris 花和修改后的 MNIST 手写数字的分类准确性从随机猜测提高到接近理论最大值，并在 MNIST、fashion-MNIST 和 Kuzushiji-MNIST 等不同数据集上展示了 AT 相对于 BP 的持续提升性能。AT 方法成功进行训练的同时，硬件开销较小，计算开销降低，是一种强大的轻量级训练替代方法，能够充分发挥物理计算的优势。