Local intrinsic dimension (LID) estimation methods have received a lot of
attention in recent years thanks to the progress in deep neural networks and
generative modeling. In opposition to old non-parametric methods, new methods
use generative models to approximate diffused dataset density and scale the
methods to high-dimensional datasets like images. In this paper, we investigate
the recent state-of-the-art parametric LID estimation methods from the
perspective of the Wiener process. We explore how these methods behave when
their assumptions are not met. We give an extended mathematical description of
those methods and their error as a function of the probability density of the
data.

本研究调查了最近最先进的参数化局部内在维度估计方法，从维纳过程的角度探讨了这些方法在其假设不满足时的行为，并给出了这些方法的扩展数学描述以及其误差与数据的概率密度函数之间的关系。

基于维纳过程的局部内在维度估计方法视角

A Wiener process perspective on local intrinsic dimension estimation  methods

Quantification of the number of variables needed to locally explain complex
data is often the first step to better understanding it. Existing techniques
from intrinsic dimension estimation leverage statistical models to glean this
information from samples within a neighborhood. However, existing methods often
rely on well-picked hyperparameters and ample data as manifold dimension and
curvature increases. Leveraging insight into the fixed point of the score
matching objective as the score map is regularized by its Dirichlet energy, we
show that it is possible to retrieve the topological dimension of the manifold
learned by the score map. We then introduce a novel method to measure the
learned manifold's topological dimension (i.e., local intrinsic dimension)
using adversarial attacks, thereby generating useful interpretations of the
learned manifold.

利用对得分映射的固定点 insight，通过利用对应的狄利克雷能量进行正则化将其前提条件确认为对潜在 manifold 学习的拓扑维度进行回归是可能的，然后引入了一种使用对抗性攻击来测量学习的 manifold 的拓扑维度的新方法，从而产生对学习的 manifold 的有用解释。

用对抗方法估计拓扑维度的谐波评分图

Adversarial Estimation of Topological Dimension with Harmonic Score Maps

One-shot neural architecture search (NAS) substantially improves the search
efficiency by training one supernet to estimate the performance of every
possible child architecture (i.e., subnet). However, the inconsistency of
characteristics among subnets incurs serious interference in the optimization,
resulting in poor performance ranking correlation of subnets. Subsequent
explorations decompose supernet weights via a particular criterion, e.g.,
gradient matching, to reduce the interference; yet they suffer from huge
computational cost and low space separability. In this work, we propose a
lightweight and effective local intrinsic dimension (LID)-based method NAS-LID.
NAS-LID evaluates the geometrical properties of architectures by calculating
the low-cost LID features layer-by-layer, and the similarity characterized by
LID enjoys better separability compared with gradients, which thus effectively
reduces the interference among subnets. Extensive experiments on NASBench-201
indicate that NAS-LID achieves superior performance with better efficiency.
Specifically, compared to the gradient-driven method, NAS-LID can save up to
86% of GPU memory overhead when searching on NASBench-201. We also demonstrate
the effectiveness of NAS-LID on ProxylessNAS and OFA spaces. Source code:
this https URL.

本文提出了一种基于局部内在维度的神经架构搜索方法 NAS-LID，有效地降低了异构子网络优化中性能排名的相关性干扰，实现了更高效的神经架构搜索。

NAS-LID：具有本地固有维度的高效神经架构搜索

NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension

Most of the existing methods for estimating the local intrinsic dimension of
a data distribution do not scale well to high-dimensional data. Many of them
rely on a non-parametric nearest neighbors approach which suffers from the
curse of dimensionality. We attempt to address that challenge by proposing a
novel approach to the problem: Local Intrinsic Dimension estimation using
approximate Likelihood (LIDL). Our method relies on an arbitrary density
estimation method as its subroutine and hence tries to sidestep the
dimensionality challenge by making use of the recent progress in parametric
neural methods for likelihood estimation. We carefully investigate the
empirical properties of the proposed method, compare them with our theoretical
predictions, and show that LIDL yields competitive results on the standard
benchmarks for this problem and that it scales to thousands of dimensions. What
is more, we anticipate this approach to improve further with the continuing
advances in the density estimation literature.

提出了一种利用极大似然法进行局部内嵌维度估计的新方法，并成功地通过利用近期参数神经方法在似然估计方面的进展，解决了对高维数据进行非参数最近邻估计中出现的问题。