In the past 20 years, artificial neural networks have become dominant in
various areas, continually growing in scale. However, the current analysis of
large models has mainly focused on functionality, overlooking the influence of
scale differences on their properties. To address this, we propose the concept
of Emergence Learning, which emphasizes the significance of scale. By studying
models of different scales, we have identified a key factor in achieving higher
performance in large models: the decrease of monosemantic neurons. Building on
this insight, we propose a proactive approach to inhibit monosemanticity for
improved performance. Our solution involves a two-phase process that includes
monosemantic neuron detection and inhibition, supported by theoretical
analysis. Experimental results on various tasks and neural networks demonstrate
the effectiveness of our proposed method.
Following the idea of Emergence Learning, though drawing inspiration from
scaling phenomena, the applicability of our method is not restricted to large
scale alone. Therefore, the experiment is self-contained. However, extending
this research to very large-scale datasets is appealing yet impossible for
research departments due to limited resources. We are delighted to share the
first co-authorship and eagerly await collaboration from any AI company before
submission.

通过研究不同规模的模型，我们发现在大型模型中达到更高性能的关键因素是单语义神经元的减少，提出了一种主动抑制单语义性的两阶段方法，并通过理论分析和实验证明了其有效性。该方法的适用性不限于大规模，但对于研究部门来说，将该研究扩展至非常大规模的数据集是吸引人的，但受到资源限制而不可能实现，期待 AI 公司的合作。

崛起学习：由新兴能力和单义性基础的研究

Emergence Learning: A Rising Direction from Emergent Abilities and a  Monosemanticity-Based Study

Local feature matching aims at establishing sparse correspondences between a
pair of images. Recently, detector-free methods present generally better
performance but are not satisfactory in image pairs with large scale
differences. In this paper, we propose Patch Area Transportation with
Subdivision (PATS) to tackle this issue. Instead of building an expensive image
pyramid, we start by splitting the original image pair into equal-sized patches
and gradually resizing and subdividing them into smaller patches with the same
scale. However, estimating scale differences between these patches is
non-trivial since the scale differences are determined by both relative camera
poses and scene structures, and thus spatially varying over image pairs.
Moreover, it is hard to obtain the ground truth for real scenes. To this end,
we propose patch area transportation, which enables learning scale differences
in a self-supervised manner. In contrast to bipartite graph matching, which
only handles one-to-one matching, our patch area transportation can deal with
many-to-many relationships. PATS improves both matching accuracy and coverage,
and shows superior performance in downstream tasks, such as relative pose
estimation, visual localization, and optical flow estimation. The source code
is available at https://zju3dv.github.io/pats/.

本文提出了一种基于分区交通的图像本地特征匹配方法，用于解决大尺度差异的图像对匹配问题。该方法不需要检测器，通过自监督学习来处理尺度差异，并能够处理多对多关系的图像匹配问题。该方法能够提高匹配精度和覆盖面，并在相对位姿估计，视觉定位和光流估计等下游任务中展现出优越性能。