Deep neural networks have been used widely to learn the latent structure of
datasets, across modalities such as images, shapes, and audio signals. However,
existing models are generally modality-dependent, requiring custom
architectures and objectives to process different classes of signals. We
leverage neural fields to capture the underlying structure in image, shape,
audio and cross-modal audiovisual domains in a modality-independent manner. We
cast our task as one of learning a manifold, where we aim to infer a
low-dimensional, locally linear subspace in which our data resides. By
enforcing coverage of the manifold, local linearity, and local isometry, our
model -- dubbed GEM -- learns to capture the underlying structure of datasets
across modalities. We can then travel along linear regions of our manifold to
obtain perceptually consistent interpolations between samples, and can further
use GEM to recover points on our manifold and glean not only diverse
completions of input images, but cross-modal hallucinations of audio or image
signals. Finally, we show that by walking across the underlying manifold of
GEM, we may generate new samples in our signal domains. Code and additional
results are available at this https URL

通过利用神经网络学习数据集的潜在结构，构建 GEM 模型，实现跨模态音像的模态无关表达与样本生成

学习神经场的信号不可知流形

Learning Signal-Agnostic Manifolds of Neural Fields

A recent advance in monolingual dependency parsing is the idea of a treebank
embedding vector, which allows all treebanks for a particular language to be
used as training data while at the same time allowing the model to prefer
training data from one treebank over others and to select the preferred
treebank at test time. We build on this idea by 1) introducing a method to
predict a treebank vector for sentences that do not come from a treebank used
in training, and 2) exploring what happens when we move away from predefined
treebank embedding vectors during test time and instead devise tailored
interpolations. We show that 1) there are interpolated vectors that are
superior to the predefined ones, and 2) treebank vectors can be predicted with
sufficient accuracy, for nine out of ten test languages, to match the
performance of an oracle approach that knows the most suitable predefined
treebank embedding for the test set.

通过引入预测句子 treebank 向量的方法以及探索在测试时移离预定义的 treebank 嵌入向量并设计 tailored 的插值方法，本研究在 monolingual dependency parsing 领域对 treebank embedding vector 的进一步改进进行了研究，并表明插值向量优于预定义的向量，对于十种语言中的九种测试语言，可以预测 treebank 向量以匹配 oracle 方法的性能。