Accurately and efficiently modeling dynamic scenes and motions is considered
so challenging a task due to temporal dynamics and motion complexity. To
address these challenges, we propose DynMF, a compact and efficient
representation that decomposes a dynamic scene into a few neural trajectories.
We argue that the per-point motions of a dynamic scene can be decomposed into a
small set of explicit or learned trajectories. Our carefully designed neural
framework consisting of a tiny set of learned basis queried only in time allows
for rendering speed similar to 3D Gaussian Splatting, surpassing 120 FPS, while
at the same time, requiring only double the storage compared to static scenes.
Our neural representation adequately constrains the inherently underconstrained
motion field of a dynamic scene leading to effective and fast optimization.
This is done by biding each point to motion coefficients that enforce the
per-point sharing of basis trajectories. By carefully applying a sparsity loss
to the motion coefficients, we are able to disentangle the motions that
comprise the scene, independently control them, and generate novel motion
combinations that have never been seen before. We can reach state-of-the-art
render quality within just 5 minutes of training and in less than half an hour,
we can synthesize novel views of dynamic scenes with superior photorealistic
quality. Our representation is interpretable, efficient, and expressive enough
to offer real-time view synthesis of complex dynamic scene motions, in
monocular and multi-view scenarios.

通过将动态场景分解为少数神经轨迹，我们提出了 DynMF 模型，该模型以高效紧凑的方式准确地建模动态场景和运动，具有实时视图合成能力。

DynMF：用于实时动态视图合成的神经运动分解与三维高斯投射

DynMF: Neural Motion Factorization for Real-time Dynamic View Synthesis  with 3D Gaussian Splatting

Understanding the neural implementation of complex human behaviors is one of
the major goals in neuroscience. To this end, it is crucial to find a true
representation of the neural data, which is challenging due to the high
complexity of behaviors and the low signal-to-ratio (SNR) of the signals. Here,
we propose a novel unsupervised learning framework, Neural Latent Aligner
(NLA), to find well-constrained, behaviorally relevant neural representations
of complex behaviors. The key idea is to align representations across repeated
trials to learn cross-trial consistent information. Furthermore, we propose a
novel, fully differentiable time warping model (TWM) to resolve the temporal
misalignment of trials. When applied to intracranial electrocorticography
(ECoG) of natural speaking, our model learns better representations for
decoding behaviors than the baseline models, especially in lower dimensional
space. The TWM is empirically validated by measuring behavioral coherence
between aligned trials. The proposed framework learns more cross-trial
consistent representations than the baselines, and when visualized, the
manifold reveals shared neural trajectories across trials.

神经学中理解复杂人类行为的神经实现是一个重要目标，本研究提出一种新的无监督学习框架，神经潜变压器（NLA），用于找到良好约束的、与行为相关的神经表示，通过对重复试验进行表示对齐来学习跨试验一致的信息，通过时间弯曲模型（TWM）解决试验之间的时间错位，应用于自然语言说话的脑内电皮层图图（ECoG）的实验证明了该模型在低维空间下学习更好的解码行为的表示，TWM 可通过测量对齐试验之间的行为一致性进行经验验证，所提出的框架学习到了比基线模型更加一致的、可视化后揭示了试验之间共享的神经轨迹的表示流形。

神经潜在校准器：用于学习复杂自然神经数据表示的跨试验校准

Neural Latent Aligner: Cross-trial Alignment for Learning  Representations of Complex, Naturalistic Neural Data

Latent variable models have become instrumental in computational neuroscience
for reasoning about neural computation. This has fostered the development of
powerful offline algorithms for extracting latent neural trajectories from
neural recordings. However, despite the potential of real time alternatives to
give immediate feedback to experimentalists, and enhance experimental design,
they have received markedly less attention. In this work, we introduce the
exponential family variational Kalman filter (eVKF), an online recursive
Bayesian method aimed at inferring latent trajectories while simultaneously
learning the dynamical system generating them. eVKF works for arbitrary
likelihoods and utilizes the constant base measure exponential family to model
the latent state stochasticity. We derive a closed-form variational analogue to
the predict step of the Kalman filter which leads to a provably tighter bound
on the ELBO compared to another online variational method. We validate our
method on synthetic and real-world data, and, notably, show that it achieves
competitive performance

本研究介绍了一种指数家族变分卡尔曼滤波器，该在线递归贝叶斯方法针对任意似然函数进行推断，利用常数基础测量指数家族对潜在状态随机性进行建模，以推断潜在轨迹并同时学习生成它们的动态系统。