This work proposes a novel face-swapping framework FlowFace++, utilizing
explicit semantic flow supervision and end-to-end architecture to facilitate
shape-aware face-swapping. Specifically, our work pretrains a facial shape
discriminator to supervise the face swapping network. The discriminator is
shape-aware and relies on a semantic flow-guided operation to explicitly
calculate the shape discrepancies between the target and source faces, thus
optimizing the face swapping network to generate highly realistic results. The
face swapping network is a stack of a pre-trained face-masked autoencoder
(MAE), a cross-attention fusion module, and a convolutional decoder. The MAE
provides a fine-grained facial image representation space, which is unified for
the target and source faces and thus facilitates final realistic results. The
cross-attention fusion module carries out the source-to-target face swapping in
a fine-grained latent space while preserving other attributes of the target
image (e.g. expression, head pose, hair, background, illumination, etc).
Lastly, the convolutional decoder further synthesizes the swapping results
according to the face-swapping latent embedding from the cross-attention fusion
module. Extensive quantitative and qualitative experiments on in-the-wild faces
demonstrate that our FlowFace++ outperforms the state-of-the-art significantly,
particularly while the source face is obstructed by uneven lighting or angle
offset.

本文提出了一种基于 FlowFace++ 的面部交换框架，利用明确的语义流监督和端到端架构来促进具有形状感知的人脸交换，并在野外人脸数据集上表现出色。

FlowFace++：显式语义流监督的端到端人脸交换

FlowFace++: Explicit Semantic Flow-supervised End-to-End Face Swapping

Casting semantic segmentation of outdoor LiDAR point clouds as a 2D problem,
e.g., via range projection, is an effective and popular approach. These
projection-based methods usually benefit from fast computations and, when
combined with techniques which use other point cloud representations, achieve
state-of-the-art results. Today, projection-based methods leverage 2D CNNs but
recent advances in computer vision show that vision transformers (ViTs) have
achieved state-of-the-art results in many image-based benchmarks. In this work,
we question if projection-based methods for 3D semantic segmentation can
benefit from these latest improvements on ViTs. We answer positively but only
after combining them with three key ingredients: (a) ViTs are notoriously hard
to train and require a lot of training data to learn powerful representations.
By preserving the same backbone architecture as for RGB images, we can exploit
the knowledge from long training on large image collections that are much
cheaper to acquire and annotate than point clouds. We reach our best results
with pre-trained ViTs on large image datasets. (b) We compensate ViTs' lack of
inductive bias by substituting a tailored convolutional stem for the classical
linear embedding layer. (c) We refine pixel-wise predictions with a
convolutional decoder and a skip connection from the convolutional stem to
combine low-level but fine-grained features of the the convolutional stem with
the high-level but coarse predictions of the ViT encoder. With these
ingredients, we show that our method, called RangeViT, outperforms existing
projection-based methods on nuScenes and SemanticKITTI. We provide the
implementation code at this https URL.

本文应用 Vision Transformers 和卷积解码器改进了基于投影的 3D 语义分割方法以获得更好的结果，并在 nuScenes 和 SemanticKITTI 数据集上超过了现有的投影方法。

RangeViT：用于自动驾驶 3D 语义分割的视觉 Transformer

RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Driving

We introduce dense vision transformers, an architecture that leverages vision
transformers in place of convolutional networks as a backbone for dense
prediction tasks. We assemble tokens from various stages of the vision
transformer into image-like representations at various resolutions and
progressively combine them into full-resolution predictions using a
convolutional decoder. The transformer backbone processes representations at a
constant and relatively high resolution and has a global receptive field at
every stage. These properties allow the dense vision transformer to provide
finer-grained and more globally coherent predictions when compared to
fully-convolutional networks. Our experiments show that this architecture
yields substantial improvements on dense prediction tasks, especially when a
large amount of training data is available. For monocular depth estimation, we
observe an improvement of up to 28% in relative performance when compared to a
state-of-the-art fully-convolutional network. When applied to semantic
segmentation, dense vision transformers set a new state of the art on ADE20K
with 49.02% mIoU. We further show that the architecture can be fine-tuned on
smaller datasets such as NYUv2, KITTI, and Pascal Context where it also sets
the new state of the art. Our models are available at
this https URL

本文提出了稠密视觉 Transformer（dense vision transformers）作为密集预测任务的主干网络，相对于全卷积网络，该结构以恒定和较高的分辨率处理表示，并在每个阶段具有全局感受野。在单眼深度估计和语义分割任务上，我们的实验表明，该结构在有大量训练数据的情况下能够显着提高性能，是一种大有前途的新型神经网络结构。