The presence of linear paths in parameter space between two different network
solutions in certain cases, i.e., linear mode connectivity (LMC), has garnered
interest from both theoretical and practical fronts. There has been significant
research that either practically designs algorithms catered for connecting
networks by adjusting for the permutation symmetries as well as some others
that more theoretically construct paths through which networks can be
connected. Yet, the core reasons for the occurrence of LMC, when in fact it
does occur, in the highly non-convex loss landscapes of neural networks are far
from clear. In this work, we take a step towards understanding it by providing
a model of how the loss landscape needs to behave topographically for LMC (or
the lack thereof) to manifest. Concretely, we present a `mountainside and
ridge' perspective that helps to neatly tie together different geometric
features that can be spotted in the loss landscape along the training runs. We
also complement this perspective by providing a theoretical analysis of the
barrier height, for which we provide empirical support, and which additionally
extends as a faithful predictor of layer-wise LMC. We close with a toy example
that provides further intuition on how barriers arise in the first place, all
in all, showcasing the larger aim of the work -- to provide a working model of
the landscape and its topography for the occurrence of LMC.

通过提供 “山坡和山脊” 视角以及理论分析障碍高度，我们致力于为线性模态连接的发生提供一个工作模型，了解在神经网络的非凸损失景观中出现线性模态连接的核心原因。