Neural fields, mapping low-dimensional input coordinates to corresponding
signals, have shown promising results in representing various signals. Numerous
methodologies have been proposed, and techniques employing MLPs and grid
representations have achieved substantial success. MLPs allow compact and high
expressibility, yet often suffer from spectral bias and slow convergence speed.
On the other hand, methods using grids are free from spectral bias and achieve
fast training speed, however, at the expense of high spatial complexity. In
this work, we propose a novel way for exploiting both MLPs and grid
representations in neural fields. Unlike the prevalent methods that combine
them sequentially (extract features from the grids first and feed them to the
MLP), we inject spectral bias-free grid representations into the intermediate
features in the MLP. More specifically, we suggest a Coordinate-Aware
Modulation (CAM), which modulates the intermediate features using scale and
shift parameters extracted from the grid representations. This can maintain the
strengths of MLPs while mitigating any remaining potential biases, facilitating
the rapid learning of high-frequency components. In addition, we empirically
found that the feature normalizations, which have not been successful in neural
filed literature, proved to be effective when applied in conjunction with the
proposed CAM. Experimental results demonstrate that CAM enhances the
performance of neural representation and improves learning stability across a
range of signals. Especially in the novel view synthesis task, we achieved
state-of-the-art performance with the least number of parameters and fast
training speed for dynamic scenes and the best performance under 1MB memory for
static scenes. CAM also outperforms the best-performing video compression
methods using neural fields by a large margin.

提出了一种利用 MLPs 和 grid representations 的新方法，通过 Coordinate-Aware Modulation (CAM) 将 grid representations 注入 MLP 的中间特征来增强神经表示的性能和学习稳定性。实验证明 CAM 在多种信号上提高了性能，并在动态场景的新视角合成任务中取得了最先进的性能，同时具有最少的参数和快速的训练速度，并且在静态场景下以 1MB 内存获得了最佳性能，相比使用神经场的最佳视频压缩方法有很大的优势。

神经场的坐标感知调制

Coordinate-Aware Modulation for Neural Fields

Tremendous progress has been made in recent years in developing better image
captioning models, yet most of them rely on a separate object detector to
extract regional features. Recent vision-language studies are shifting towards
the detector-free trend by leveraging grid representations for more flexible
model training and faster inference speed. However, such development is
primarily focused on image understanding tasks, and remains less investigated
for the caption generation task. In this paper, we are concerned with a
better-performing detector-free image captioning model, and propose a pure
vision transformer-based image captioning model, dubbed as ViTCAP, in which
grid representations are used without extracting the regional features. For
improved performance, we introduce a novel Concept Token Network (CTN) to
predict the semantic concepts and then incorporate them into the end-to-end
captioning. In particular, the CTN is built on the basis of a vision
transformer and is designed to predict the concept tokens through a
classification task, from which the rich semantic information contained greatly
benefits the captioning task. Compared with the previous detector-based models,
ViTCAP drastically simplifies the architectures and at the same time achieves
competitive performance on various challenging image captioning datasets. In
particular, ViTCAP reaches 138.1 CIDEr scores on COCO-caption Karpathy-split,
93.8 and 108.6 CIDEr scores on nocaps, and Google-CC captioning datasets,
respectively.

本文提出了一种全新的视觉变换器（Vision transformer-based）图像标题模型 ViTCAP，使用网格表示而不提取区域特征。为了提高性能，引入了基于 Vision Transformer 的概念令牌网络（CTN），可以高效地预测语义概念，并将其整合到端到端的标题生成中，从而实现竞争性能，同时简化了结构。

将语义概念注入端到端图像字幕生成

Injecting Semantic Concepts into End-to-End Image Captioning

Forecasting long-term human motion is a challenging task due to the
non-linearity, multi-modality and inherent uncertainty in future trajectories.
The underlying scene and past motion of agents can provide useful cues to
predict their future motion. However, the heterogeneity of the two inputs poses
a challenge for learning a joint representation of the scene and past
trajectories. To address this challenge, we propose a model based on grid
representations to forecast agent trajectories. We represent the past
trajectories of agents using binary 2-D grids, and the underlying scene as a
RGB birds-eye view (BEV) image, with an agent-centric frame of reference. We
encode the scene and past trajectories using convolutional layers and generate
trajectory forecasts using a Convolutional LSTM (ConvLSTM) decoder. Results on
the publicly available Stanford Drone Dataset (SDD) show that our model
outperforms prior approaches and outputs realistic future trajectories that
comply with scene structure and past motion.

使用基于网格表示的模型，结合卷积神经网络及 ConvLSTM 解码器，针对人体运动预测中存在的非线性、多模态和不确定性等问题提出了一种解决方案，取得了优于现有方法的效果和真实性。