We present the content deformation field CoDeF as a new type of video
representation, which consists of a canonical content field aggregating the
static contents in the entire video and a temporal deformation field recording
the transformations from the canonical image (i.e., rendered from the canonical
content field) to each individual frame along the time axis.Given a target
video, these two fields are jointly optimized to reconstruct it through a
carefully tailored rendering pipeline.We advisedly introduce some
regularizations into the optimization process, urging the canonical content
field to inherit semantics (e.g., the object shape) from the video.With such a
design, CoDeF naturally supports lifting image algorithms for video processing,
in the sense that one can apply an image algorithm to the canonical image and
effortlessly propagate the outcomes to the entire video with the aid of the
temporal deformation field.We experimentally show that CoDeF is able to lift
image-to-image translation to video-to-video translation and lift keypoint
detection to keypoint tracking without any training.More importantly, thanks to
our lifting strategy that deploys the algorithms on only one image, we achieve
superior cross-frame consistency in processed videos compared to existing
video-to-video translation approaches, and even manage to track non-rigid
objects like water and smog.Project page can be found at
this https URL

通过引入内容变形场 CoDeF 作为新型视频表示方式，可以将图像算法应用于视频处理，并具有超过现有视频转换方法的帧间一致性。

CoDeF: 内容变形字段用于时间一致的视频处理

CoDeF: Content Deformation Fields for Temporally Consistent Video  Processing

Neural radiance fields (NeRFs) are able to synthesize realistic novel views
from multi-view images captured from distinct positions and perspectives. In
NeRF's rendering pipeline, neural networks are used to represent a scene
independently or transform queried learnable feature vector of a point to the
expected color or density. With the aid of geometry guides either in occupancy
grids or proposal networks, the number of neural network evaluations can be
reduced from hundreds to dozens in the standard volume rendering framework.
Instead of rendering yielded color after neural network evaluation, we propose
to render the queried feature vectors of a ray first and then transform the
rendered feature vector to the final pixel color by a neural network. This
fundamental change to the standard volume rendering framework requires only one
single neural network evaluation to render a pixel, which substantially lowers
the high computational complexity of the rendering framework attributed to a
large number of neural network evaluations. Consequently, we can use a
comparably larger neural network to achieve a better rendering quality while
maintaining the same training and rendering time costs. Our model achieves the
state-of-the-art rendering quality on both synthetic and real-world datasets
while requiring a training time of several minutes.

本研究提出了将本应在神经网络评估后再进行渲染的颜色，改为直接渲染出射线查询的特征向量，从而大幅降低了神经网络评估的复杂度和计算开销，并取得了较好的渲染质量。

快速神经辐射场重建的体积特征渲染

Volume Feature Rendering for Fast Neural Radiance Field Reconstruction

State-of-the-art face recognition models show impressive accuracy, achieving
over 99.8% on Labeled Faces in the Wild (LFW) dataset. Such models are trained
on large-scale datasets that contain millions of real human face images
collected from the internet. Web-crawled face images are severely biased (in
terms of race, lighting, make-up, etc) and often contain label noise. More
importantly, the face images are collected without explicit consent, raising
ethical concerns. To avoid such problems, we introduce a large-scale synthetic
dataset for face recognition, obtained by rendering digital faces using a
computer graphics pipeline. We first demonstrate that aggressive data
augmentation can significantly reduce the synthetic-to-real domain gap. Having
full control over the rendering pipeline, we also study how each attribute
(e.g., variation in facial pose, accessories and textures) affects the
accuracy. Compared to SynFace, a recent method trained on GAN-generated
synthetic faces, we reduce the error rate on LFW by 52.5% (accuracy from 91.93%
to 96.17%). By fine-tuning the network on a smaller number of real face images
that could reasonably be obtained with consent, we achieve accuracy that is
comparable to the methods trained on millions of real face images.

采用合成数据集进行人脸识别，通过数据扩充和精细控制属性可以显著减少真实与合成数据集之间的差异，并降低数据集的偏见和标签噪声问题，从而将识别错误率从 91.93% 降低到 96.17%，并在较少的真实样本上训练可与百万真实样本训练相媲美的识别效果。