The expressivity of Graph Neural Networks (GNNs) can be entirely
characterized by appropriate fragments of the first order logic. Namely, any
query of the two variable fragment of graded modal logic (GC2) interpreted over
labelled graphs can be expressed using a GNN whose size depends only on the
depth of the query. As pointed out by [Barcelo & Al., 2020, Grohe, 2021 ], this
description holds for a family of activation functions, leaving the
possibibility for a hierarchy of logics expressible by GNNs depending on the
chosen activation function. In this article, we show that such hierarchy indeed
exists by proving that GC2 queries cannot be expressed by GNNs with polynomial
activation functions. This implies a separation between polynomial and popular
non polynomial activations (such as ReLUs, sigmoid and hyperbolic tan and
others) and answers an open question formulated by [Grohe, 2021].

图神经网络的表达能力可以完全用合适的一阶逻辑片段来描述。本文通过证明 GC2 查询不能由具有多项式激活函数的图神经网络表示，证明了多项式和流行的非多项式激活函数之间存在分离关系。

多项式激活函数的图神经网络具有有限表达性

Graph Neural Networks with polynomial activations have limited  expressivity

We study deep neural networks with polynomial activations, particularly their
expressive power. For a fixed architecture and activation degree, a polynomial
neural network defines an algebraic map from weights to polynomials. The image
of this map is the functional space associated to the network, and it is an
irreducible algebraic variety upon taking closure. This paper proposes the
dimension of this variety as a precise measure of the expressive power of
polynomial neural networks. We obtain several theoretical results regarding
this dimension as a function of architecture, including an exact formula for
high activation degrees, as well as upper and lower bounds on layer widths in
order for deep polynomials networks to fill the ambient functional space. We
also present computational evidence that it is profitable in terms of
expressiveness for layer widths to increase monotonically and then decrease
monotonically. Finally, we link our study to favorable optimization properties
when training weights, and we draw intriguing connections with tensor and
polynomial decompositions.

通过研究多项式激活的深度神经网络，我们提出了 “维度” 作为多项式神经网络表现力的度量标准，并探讨了它受体系结构影响的理论结果。同时，我们还将我们的研究与有利的优化性质联系起来，以及与张量和多项式分解等领域产生了有趣的关联。