Generalizable implicit neural representation (INR) enables a single
continuous function, i.e., a coordinate-based neural network, to represent
multiple data instances by modulating its weights or intermediate features
using latent codes. However, the expressive power of the state-of-the-art
modulation is limited due to its inability to localize and capture fine-grained
details of data entities such as specific pixels and rays. To address this
issue, we propose a novel framework for generalizable INR that combines a
transformer encoder with a locality-aware INR decoder. The transformer encoder
predicts a set of latent tokens from a data instance to encode local
information into each latent token. The locality-aware INR decoder extracts a
modulation vector by selectively aggregating the latent tokens via
cross-attention for a coordinate input and then predicts the output by
progressively decoding with coarse-to-fine modulation through multiple
frequency bandwidths. The selective token aggregation and the multi-band
feature modulation enable us to learn locality-aware representation in spatial
and spectral aspects, respectively. Our framework significantly outperforms
previous generalizable INRs and validates the usefulness of the locality-aware
latents for downstream tasks such as image generation.

提出了一种新的通用的内隐神经表示（INR）框架，结合了 transformer 编码器和可感知局部性的 INR 解码器，显著优于以往的通用的 INR，并验证了局部感知潜变量在图像生成等下游任务中的有效性。