The representation of arbitrary data in a biological system is one of the
most elusive elements of biological information processing. The often
logarithmic nature of information in amplitude and frequency presented to
biosystems prevents simple encapsulation of the information contained in the
input. Criticality Analysis (CA) is a bio-inspired method of information
representation within a controlled self-organised critical system that allows
scale-free representation. This is based on the concept of a reservoir of
dynamic behaviour in which self-similar data will create dynamic nonlinear
representations. This unique projection of data preserves the similarity of
data within a multidimensional neighbourhood. The input can be reduced
dimensionally to a projection output that retains the features of the overall
data, yet has much simpler dynamic response. The method depends only on the
rate control of chaos applied to the underlying controlled models, that allows
the encoding of arbitrary data, and promises optimal encoding of data given
biological relevant networks of oscillators. The CA method allows for a
biologically relevant encoding mechanism of arbitrary input to biosystems,
creating a suitable model for information processing in varying complexity of
organisms and scale-free data representation for machine learning.

本文提出基于临界分析的新型信息编码方法，利用自组织临界系统中的动态行为来创建非线性、尺度自由的数据表示，这种数据投影方法可以将输入数据的特性保留在降维后的输出中，提供了适用于生物系统的编码机制以及机器学习的尺度自由数据表示。

临界性分析：生物启发的非线性数据表示

Criticality Analysis: Bio-inspired Nonlinear Data Representation

Neural Ordinary Differential Equations (NODEs), a framework of
continuous-depth neural networks, have been widely applied, showing exceptional
efficacy in coping with representative datasets. Recently, an augmented
framework has been developed to overcome some limitations that emerged in the
application of the original framework. In this paper, we propose a new class of
continuous-depth neural networks with delay, named Neural Delay Differential
Equations (NDDEs). To compute the corresponding gradients, we use the adjoint
sensitivity method to obtain the delayed dynamics of the adjoint. Differential
equations with delays are typically seen as dynamical systems of infinite
dimension that possess more fruitful dynamics. Compared to NODEs, NDDEs have a
stronger capacity of nonlinear representations. We use several illustrative
examples to demonstrate this outstanding capacity. Firstly, we successfully
model the delayed dynamics where the trajectories in the lower-dimensional
phase space could be mutually intersected and even chaotic in a model-free or
model-based manner. Traditional NODEs, without any argumentation, are not
directly applicable for such modeling. Secondly, we achieve lower loss and
higher accuracy not only for the data produced synthetically by complex models
but also for the CIFAR10, a well-known image dataset. Our results on the NDDEs
demonstrate that appropriately articulating the elements of dynamical systems
into the network design is truly beneficial in promoting network performance.

本文介绍了一种名为 Neural Delay Differential Equations（NDDE）的连续深度神经网络，使用输入的延迟动态学方程计算相应的梯度，并用数个实际案例展示了 NDDE 比传统模型具有更强的非线性表达能力和性能表现的优势。

神经时滞微分方程：系统重构与图像分类

Neural Delay Differential Equations: System Reconstruction and Image  Classification

Can we effectively learn a nonlinear representation in time comparable to
linear learning? We describe a new algorithm that explicitly and adaptively
expands higher-order interaction features over base linear representations. The
algorithm is designed for extreme computational efficiency, and an extensive
experimental study shows that its computation/prediction tradeoff ability
compares very favorably against strong baselines.

我们描述了一种新的算法，可以在计算效率和预测性能之间取得良好的平衡，该算法可以针对非线性表达式进行高效计算，并扩展高阶交互特征。