Modern numerical analysis is executed on discrete data, of which numerical
difference computation is one of the cores and is indispensable. Nevertheless,
difference algorithms have a critical weakness in their sensitivity to noise,
which has long posed a challenge in various fields including signal processing.
Difference is an extension or generalization of differential in the discrete
domain. However, due to the finite interval in discrete calculation, there is a
failure in meeting the most fundamental definition of differential, where dy
and dx are both infinitesimal (Leibniz) or the limit of dx is 0 (Cauchy). In
this regard, the generalization of differential to difference does not hold. To
address this issue, we depart from the original derivative approach, construct
a finite interval-based differential, and further generalize it to obtain the
difference by convolution. Based on this theory, we present a variety of
difference operators suitable for practical signal processing. Experimental
results demonstrate that these difference operators possess exceptional signal
processing capabilities, including high noise immunity.

本文介绍了一种基于有限区间微分的离散信号处理方法，通过对微分方法进行推广并结合卷积，提出了一系列方便实际操作且具有高抗噪性能的差分算子。

Tao 通用微分与差分：理论与应用

Tao General Differential and Difference: Theory and Application

Spiking neural networks (SNNs) are well known as the brain-inspired models
with high computing efficiency, due to a key component that they utilize spikes
as information units, close to the biological neural systems. Although spiking
based models are energy efficient by taking advantage of discrete spike
signals, their performance is limited by current network structures and their
training methods. As discrete signals, typical SNNs cannot apply the gradient
descent rules directly into parameters adjustment as artificial neural networks
(ANNs). Aiming at this limitation, here we propose a novel method of
constructing deep SNN models with knowledge distillation (KD) that uses ANN as
teacher model and SNN as student model. Through ANN-SNN joint training
algorithm, the student SNN model can learn rich feature information from the
teacher ANN model through the KD method, yet it avoids training SNN from
scratch when communicating with non-differentiable spikes. Our method can not
only build a more efficient deep spiking structure feasibly and reasonably, but
use few time steps to train whole model compared to direct training or ANN to
SNN methods. More importantly, it has a superb ability of noise immunity for
various types of artificial noises and natural signals. The proposed novel
method provides efficient ways to improve the performance of SNN through
constructing deeper structures in a high-throughput fashion, with potential
usage for light and efficient brain-inspired computing of practical scenarios.

本文提出了一种新的方法，利用知识蒸馏构建深度 SNN 模型，通过利用 ANN 作为教师模型和 SNN 作为学生模型进行联合训练算法，可以更加有效地构建深层脉冲神经网络结构，并且在与不可微分波形的通信时，仍然能够避免从零开始训练 SNN，并且具有良好的噪声免疫能力。