Many settings in machine learning require the selection of a rotation
representation. However, choosing a suitable representation from the many
available options is challenging. This paper acts as a survey and guide through
rotation representations. We walk through their properties that harm or benefit
deep learning with gradient-based optimization. By consolidating insights from
rotation-based learning, we provide a comprehensive overview of learning
functions with rotation representations. We provide guidance on selecting
representations based on whether rotations are in the model's input or output
and whether the data primarily comprises small angles.

这篇论文综述了旋转表示的选择问题，阐述了其在深度学习与基于梯度优化的问题中的优劣性，提供了基于输入输出是否包含旋转以及数据是否主要由小角度构成的情况下选择合适表示的指导。

学习与三维旋转，SO (3) 的自助指南

Learning with 3D rotations, a hitchhiker's guide to SO(3)

We consider the problem of learning functions in the $\mathcal{F}_{p,\pi}$
and Barron spaces, which are natural function spaces that arise in the
high-dimensional analysis of random feature models (RFMs) and two-layer neural
networks. Through a duality analysis, we reveal that the approximation and
estimation of these spaces can be considered equivalent in a certain sense.
This enables us to focus on the easier problem of approximation and estimation
when studying the generalization of both models. The dual equivalence is
established by defining an information-based complexity that can effectively
control estimation errors. Additionally, we demonstrate the flexibility of our
duality framework through comprehensive analyses of two concrete applications.
The first application is to study learning functions in $\mathcal{F}_{p,\pi}$
with RFMs. We prove that the learning does not suffer from the curse of
dimensionality as long as $p>1$, implying RFMs can work beyond the kernel
regime. Our analysis extends existing results [CMM21] to the noisy case and
removes the requirement of overparameterization.
The second application is to investigate the learnability of reproducing
kernel Hilbert space (RKHS) under the $L^\infty$ metric. We derive both lower
and upper bounds of the minimax estimation error by using the spectrum of the
associated kernel. We then apply these bounds to dot-product kernels and
analyze how they scale with the input dimension. Our results suggest that
learning with ReLU (random) features is generally intractable in terms of
reaching high uniform accuracy.

本研究通过对学习 Fp,π 和 Barron 空间中的函数这一问题的探讨，揭示了这些空间的逼近和估计可以在某种意义上等价。这使我们能够将重点放在逼近和估计这一较容易的问题上，当研究两个模型的泛化时。此外，我们通过两个具体的应用程序全面分析了我们的二元性框架的灵活性。

一种对随机特征模型和两层神经网络进行泛化分析的二元框架

A duality framework for generalization analysis of random feature models  and two-layer neural networks

An intriguing phenomenon observed during training neural networks is the
spectral bias, which states that neural networks are biased towards learning
less complex functions. The priority of learning functions with low complexity
might be at the core of explaining generalization ability of neural network,
and certain efforts have been made to provide theoretical explanation for
spectral bias. However, there is still no satisfying theoretical result
justifying the underlying mechanism of spectral bias. In this paper, we give a
comprehensive and rigorous explanation for spectral bias and relate it with the
neural tangent kernel function proposed in recent work. We prove that the
training process of neural networks can be decomposed along different
directions defined by the eigenfunctions of the neural tangent kernel, where
each direction has its own convergence rate and the rate is determined by the
corresponding eigenvalue. We then provide a case study when the input data is
uniformly distributed over the unit sphere, and show that lower degree
spherical harmonics are easier to be learned by over-parameterized neural
networks. Finally, we provide numerical experiments to demonstrate the
correctness of our theory. Our experimental results also show that our theory
can tolerate certain model misspecification in terms of the input data
distribution.

论文通过与神经切向核函数的关联性进行解释，给出了关于神经网络中谱偏置的全面而严格的解释，并证明了神经网络的训练过程可以沿神经切向核的不同方向分解，每个方向都有自己的收敛率，而该收敛率由相应的特征值确定。