Recent work has suggested that certain neural network architectures-particularly recurrent neural networks (RNNs) and implicit neural networks (INNs) are capable of logical extrapolation. That is, one may train such a network on easy instances of a specific task and then apply it successfully to more difficult instances of the same task. In this paper, we revisit this idea and show that (i) The capacity for extrapolation is less robust than previously suggested. Specifically, in the context of a maze-solving task, we show that while INNs (and some RNNs) are capable of generalizing to larger maze instances, they fail to generalize along axes of difficulty other than maze size. (ii) Models that are explicitly trained to converge to a fixed point (e.g. the INN we test) are likely to do so when extrapolating, while models that are not (e.g. the RNN we test) may exhibit more exotic limiting behaviour such as limit cycles, even when they correctly solve the problem. Our results suggest that (i) further study into why such networks extrapolate easily along certain axes of difficulty yet struggle with others is necessary, and (ii) analyzing the dynamics of extrapolation may yield insights into designing more efficient and interpretable logical extrapolators.

本研究探讨了递归神经网络（RNNs）和隐式神经网络（INNs）在逻辑外推中的能力，特别是在迷宫求解任务中，发现这些网络在大小方面能够泛化，但在其他困难维度上有所欠缺。此外，研究表明，训练收敛于固定点的模型在外推时表现良好，而未针对固定点训练的模型则可能展现出更复杂的极限行为。这强调了对外推动态的深入分析对于设计更高效且可解释的逻辑外推器的重要性。

关于具有递归和隐式网络的迷宫逻辑外推