Motivated by the growing theoretical understanding of neural networks that
employ the Rectified Linear Unit (ReLU) as their activation function, we
revisit the use of ReLU activation functions for learning implicit neural
representations (INRs). Inspired by second order B-spline wavelets, we
incorporate a set of simple constraints to the ReLU neurons in each layer of a
deep neural network (DNN) to remedy the spectral bias. This in turn enables its
use for various INR tasks. Empirically, we demonstrate that, contrary to
popular belief, one can learn state-of-the-art INRs based on a DNN composed of
only ReLU neurons. Next, by leveraging recent theoretical works which
characterize the kinds of functions ReLU neural networks learn, we provide a
way to quantify the regularity of the learned function. This offers a
principled approach to selecting the hyperparameters in INR architectures. We
substantiate our claims through experiments in signal representation, super
resolution, and computed tomography, demonstrating the versatility and
effectiveness of our method. The code for all experiments can be found at
this https URL

通过第二阶 B 样条小波的灵感，我们给深度神经网络 (DNN) 每一层的 ReLU 神经元引入一组简单的约束以修正其谱偏差，从而实现对各种隐式神经表示任务的有效使用。通过实验证明，与普遍观点相反，基于只包含 ReLU 神经元的 DNN 可以学习出最先进的隐式神经表示。我们利用最近关于 ReLU 神经网络学习函数类型的理论工作，提供了一种量化学习函数规则性的方法，为 INR 架构中的超参数选择提供了有原则的方法。通过在信号表示、超分辨率和计算机断层扫描等领域进行实验，我们验证了我们方法的多样性和有效性。所有实验的代码可以在此 URL 中找到。