We consider a binary classifier defined as the sign of a tropical rational
function, that is, as the difference of two convex piecewise linear functions.
The parameter space of ReLU neural networks is contained as a semialgebraic set
inside the parameter space of tropical rational functions. We initiate the
study of two different subdivisions of this parameter space: a subdivision into
semialgebraic sets, on which the combinatorial type of the decision boundary is
fixed, and a subdivision into a polyhedral fan, capturing the combinatorics of
the partitions of the dataset. The sublevel sets of the 0/1-loss function arise
as subfans of this classification fan, and we show that the level-sets are not
necessarily connected. We describe the classification fan i) geometrically, as
normal fan of the activation polytope, and ii) combinatorially through a list
of properties of associated bipartite graphs, in analogy to covector axioms of
oriented matroids and tropical oriented matroids. Our findings extend and
refine the connection between neural networks and tropical geometry by
observing structures established in real tropical geometry, such as positive
tropicalizations of hypersurfaces and tropical semialgebraic sets.

考虑一个由两个凸分段线性函数差值定义的二元分类器，其中 ReLU 神经网络的参数空间包含在热带有理函数的参数空间中，我们将该参数空间划分为两个不同的子集：一个由半代数集组成，其决策边界的组合类型固定；一个由多面体扇形组成，捕捉了数据集分割的组合数学。0/1 损失函数的亚水平集是分类扇形的子扇，我们证明亚水平集不一定是连通的。我们从几何学上描述了分类扇形，作为激活多面体的法向扇形，从组合学上通过关联的二分图列表描述了该扇形，类似于有向拓扑和热带有向拓扑的对偶向量公理。我们的发现通过观察在实际热带几何中建立的结构扩展和改进了神经网络与热带几何之间的联系，如超曲面的正热带化和热带半代数集。

神经网络的实际热带几何

The Real Tropical Geometry of Neural Networks

We propose an alternating minimization heuristic for regression over the
space of tropical rational functions with fixed exponents. The method
alternates between fitting the numerator and denominator terms via tropical
polynomial regression, which is known to admit a closed form solution. We
demonstrate the behavior of the alternating minimization method experimentally.
Experiments demonstrate that the heuristic provides a reasonable approximation
of the input data. Our work is motivated by applications to ReLU neural
networks, a popular class of network architectures in the machine learning
community which are closely related to tropical rational functions.

提出了一种用于回归的交替最小化启发式算法，其在固定指数的热带有理函数空间上进行。该方法通过热带多项式回归逐次拟合分子和分母项，并已在实验中展示了其行为。该启发式方法提供了输入数据的合理近似，其工作受到与热带有理函数密切相关的流行机器学习网络结构 ReLU 神经网络的应用的启发。