Utilizing multi-view inputs to synthesize novel-view images, Neural Radiance
Fields (NeRF) have emerged as a popular research topic in 3D vision. In this
work, we introduce a Generalizable Semantic Neural Radiance Field (GSNeRF),
which uniquely takes image semantics into the synthesis process so that both
novel view images and the associated semantic maps can be produced for unseen
scenes. Our GSNeRF is composed of two stages: Semantic Geo-Reasoning and
Depth-Guided Visual rendering. The former is able to observe multi-view image
inputs to extract semantic and geometry features from a scene. Guided by the
resulting image geometry information, the latter performs both image and
semantic rendering with improved performances. Our experiments not only confirm
that GSNeRF performs favorably against prior works on both novel-view image and
semantic segmentation synthesis but the effectiveness of our sampling strategy
for visual rendering is further verified.

使用多视图输入合成新视图图像，神经辐射场（NeRF）已成为三维视觉中的热门研究课题。本文提出了一种具有一般化语义神经辐射场（GSNeRF）的方法，该方法独特地将图像语义纳入合成过程中，从而能够对未见场景生成新视图图像和相关的语义地图。我们的 GSNeRF 由两个阶段组成：语义地理推理和深度引导的视觉渲染。前者能够从场景中观察多视图图像输入，提取语义和几何特征。在后者的指导下，利用得到的图像几何信息，进行图像和语义渲染，从而提高性能。我们的实验证实了 GSNeRF 在新视图图像和语义分割合成方面优于先前的工作，并进一步验证了我们的采样策略的有效性。

GSNeRF: 增强的通用语义神经辐射场与 3D 场景理解

GSNeRF: Generalizable Semantic Neural Radiance Fields with Enhanced 3D  Scene Understanding

Automating sleep staging is vital to scale up sleep assessment and diagnosis
to serve millions experiencing sleep deprivation and disorders and enable
longitudinal sleep monitoring in home environments. Learning from raw
polysomnography signals and their derived time-frequency image representations
has been prevalent. However, learning from multi-view inputs (e.g., both the
raw signals and the time-frequency images) for sleep staging is difficult and
not well understood. This work proposes a sequence-to-sequence sleep staging
model, XSleepNet, that is capable of learning a joint representation from both
raw signals and time-frequency images. Since different views may generalize or
overfit at different rates, the proposed network is trained such that the
learning pace on each view is adapted based on their generalization/overfitting
behavior. In simple terms, the learning on a particular view is speeded up when
it is generalizing well and slowed down when it is overfitting. View-specific
generalization/overfitting measures are computed on-the-fly during the training
course and used to derive weights to blend the gradients from different views.
As a result, the network is able to retain the representation power of
different views in the joint features which represent the underlying
distribution better than those learned by each individual view alone.
Furthermore, the XSleepNet architecture is principally designed to gain
robustness to the amount of training data and to increase the complementarity
between the input views. Experimental results on five databases of different
sizes show that XSleepNet consistently outperforms the single-view baselines
and the multi-view baseline with a simple fusion strategy. Finally, XSleepNet
also outperforms prior sleep staging methods and improves previous
state-of-the-art results on the experimental databases.

本文提出了一个名为 XSleepNet 的序列到序列的睡眠分期模型，可以从不同的视角（例如，原始信号和时频图像）学习联合表示，在不同视角之间平衡表示功率，从而改善了睡眠分期方法的结果。