In pioneering work from 2019, Barcel\'o and coauthors identified logics that
precisely match the expressive power of constant iteration-depth graph neural
networks (GNNs) relative to properties definable in first-order logic. In this
article, we give exact logical characterizations of recurrent GNNs in two
scenarios: (1) in the setting with floating-point numbers and (2) with reals.
For floats, the formalism matching recurrent GNNs is a rule-based modal logic
with counting, while for reals we use a suitable infinitary modal logic, also
with counting. These results give exact matches between logics and GNNs in the
recurrent setting without relativising to a background logic in either case,
but using some natural assumptions about floating-point arithmetic. Applying
our characterizations, we also prove that, relative to graph properties
definable in monadic second-order logic (MSO), our infinitary and rule-based
logics are equally expressive. This implies that recurrent GNNs with reals and
floats have the same expressive power over MSO-definable properties and shows
that, for such properties, also recurrent GNNs with reals are characterized by
a (finitary!) rule-based modal logic. In the general case, in contrast, the
expressive power with floats is weaker than with reals. In addition to
logic-oriented results, we also characterize recurrent GNNs, with both reals
and floats, via distributed automata, drawing links to distributed computing
models.

对于具有浮点数和实数的循环图神经网络，我们给出了两种情况下的精确逻辑特征，对于浮点数，与循环 GNN 匹配的形式是一种具有计数的基于规则的模态逻辑，而对于实数，我们使用适当的无限模态逻辑，也带有计数。此外，相对于在单调二阶逻辑（MSO）中定义的图属性，我们的无限和基于规则的逻辑的表达力是相等的，这意味着具有实数和浮点数的循环 GNN 在 MSO 可定义的属性上具有相同的表达力，并且说明对于这样的属性，实数循环 GNN 也是由（有限的！）基于规则的模态逻辑来特征化的。

具有实数和浮点数的循环图神经网络的逻辑特征

Logical Characterizations of Recurrent Graph Neural Networks with Reals  and Floats

The study of the expressive power of neural networks has investigated the
fundamental limits of neural networks. Most existing results assume real-valued
inputs and parameters as well as exact operations during the evaluation of
neural networks. However, neural networks are typically executed on computers
that can only represent a tiny subset of the reals and apply inexact
operations. In this work, we analyze the expressive power of neural networks
under a more realistic setup: when we use floating-point numbers and
operations. Our first set of results assumes floating-point operations where
the significand of a float is represented by finite bits but its exponent can
take any integer value. Under this setup, we show that neural networks using a
binary threshold unit or ReLU can memorize any finite input/output pairs and
can approximate any continuous function within a small error. We also show
similar results on memorization and universal approximation when floating-point
operations use finite bits for both significand and exponent; these results are
applicable to many popular floating-point formats such as those defined in the
IEEE 754 standard (e.g., 32-bit single-precision format) and bfloat16.

利用浮点数和操作，研究了神经网络的表达能力，得出了使用二进制阈值单元或 ReLU 的神经网络可以记忆任何有限的输入 / 输出对并且可以在误差范围内近似任何连续函数的结论。

ReLU 与 Step 网络在浮点运算下的表现能力

Expressive Power of ReLU and Step Networks under Floating-Point  Operations

Deep convolutional neural network (CNN) inference requires significant amount
of memory and computation, which limits its deployment on embedded devices. To
alleviate these problems to some extent, prior research utilize low precision
fixed-point numbers to represent the CNN weights and activations. However, the
minimum required data precision of fixed-point weights varies across different
networks and also across different layers of the same network. In this work, we
propose using floating-point numbers for representing the weights and
fixed-point numbers for representing the activations. We show that using
floating-point representation for weights is more efficient than fixed-point
representation for the same bit-width and demonstrate it on popular large-scale
CNNs such as AlexNet, SqueezeNet, GoogLeNet and VGG-16. We also show that such
a representation scheme enables compact hardware multiply-and-accumulate (MAC)
unit design. Experimental results show that the proposed scheme reduces the
weight storage by up to 36% and power consumption of the hardware multiplier by
up to 50%.

本文提出使用浮点数表示权重和使用定点数表示激活，证明了相较于定点数表示，使用浮点数表示权重更加高效，同时能够优化硬件乘加器设计，实验表明这个方法最多可减少 36% 的权重存储和 50% 的硬件乘加器功耗。