We present a neural network architecture designed to naturally learn a
positional embedding and overcome the spectral bias towards lower frequencies
faced by conventional implicit neural representation networks. Our proposed
architecture, SPDER, is a simple MLP that uses an activation function composed
of a sinusoidal multiplied by a sublinear function, called the damping
function. The sinusoidal enables the network to automatically learn the
positional embedding of an input coordinate while the damping passes on the
actual coordinate value by preventing it from being projected down to within a
finite range of values. Our results indicate that SPDERs speed up training by
10x and converge to losses 1,500-50,000x lower than that of the
state-of-the-art for image representation. SPDER is also state-of-the-art in
audio representation. The superior representation capability allows SPDER to
also excel on multiple downstream tasks such as image super-resolution and
video frame interpolation. We provide intuition as to why SPDER significantly
improves fitting compared to that of other INR methods while requiring no
hyperparameter tuning or preprocessing.

文章提出了一种称为 SPDER 的简单多层感知器神经网络结构，能够自然地学习位置嵌入，克服了传统隐式神经表示网络面临的较低频率谱偏差问题，并且速度很快、收敛到极低的损失水平，并且在图像表示中表现出与众不同的表现能力。