Neural Radiance Fields (NeRF) revolutionize the realm of visual media by
providing photorealistic Free-Viewpoint Video (FVV) experiences, offering
viewers unparalleled immersion and interactivity. However, the technology's
significant storage requirements and the computational complexity involved in
generation and rendering currently limit its broader application. To close this
gap, this paper presents Temporal Tri-Plane Radiance Fields (TeTriRF), a novel
technology that significantly reduces the storage size for Free-Viewpoint Video
(FVV) while maintaining low-cost generation and rendering. TeTriRF introduces a
hybrid representation with tri-planes and voxel grids to support scaling up to
long-duration sequences and scenes with complex motions or rapid changes. We
propose a group training scheme tailored to achieving high training efficiency
and yielding temporally consistent, low-entropy scene representations.
Leveraging these properties of the representations, we introduce a compression
pipeline with off-the-shelf video codecs, achieving an order of magnitude less
storage size compared to the state-of-the-art. Our experiments demonstrate that
TeTriRF can achieve competitive quality with a higher compression rate.

本研究提出了 Temporoal Tri-Plane Radiance Fields（TeTriRF），通过引入三平面和体素网格的混合表示方法，支持长时序列和场景的大规模扩展，并使用现成的视频编解码器，实现与现有技术相比一个数量级的存储空间减小，同时保持高压缩率和竞争性的质量。

TeTriRF：用于高效自由视角视频的时域三平面辐射场

TeTriRF: Temporal Tri-Plane Radiance Fields for Efficient Free-Viewpoint  Video

Recently, Vision Transformers (ViTs) have attracted a lot of attention in the
field of computer vision. Generally, the powerful representative capacity of
ViTs mainly benefits from the self-attention mechanism, which has a high
computation complexity. To accelerate ViTs, we propose an integrated
compression pipeline based on observed heterogeneous attention patterns across
layers. On one hand, different images share more similar attention patterns in
early layers than later layers, indicating that the dynamic query-by-key
self-attention matrix may be replaced with a static self-attention matrix in
early layers. Then, we propose a dynamic-guided static self-attention (DGSSA)
method where the matrix inherits self-attention information from the replaced
dynamic self-attention to effectively improve the feature representation
ability of ViTs. On the other hand, the attention maps have more low-rank
patterns, which reflect token redundancy, in later layers than early layers. In
a view of linear dimension reduction, we further propose a method of global
aggregation pyramid (GLAD) to reduce the number of tokens in later layers of
ViTs, such as Deit. Experimentally, the integrated compression pipeline of
DGSSA and GLAD can accelerate up to 121% run-time throughput compared with
DeiT, which surpasses all SOTA approaches.

基于观察到的异构注意力模式，在 Vision Transformers 中提出了一种综合压缩方法，通过动态引导的静态自注意力和全局聚合金字塔，加速了运行时间吞吐量，超过了所有 SOTA 方法。

基于异质注意力模式的视觉 Transformer 加速

Accelerating Vision Transformers Based on Heterogeneous Attention  Patterns

Model compression is a critical technique to efficiently deploy neural
network models on mobile devices which have limited computation resources and
tight power budgets. Conventional model compression techniques rely on
hand-crafted heuristics and rule-based policies that require domain experts to
explore the large design space trading off among model size, speed, and
accuracy, which is usually sub-optimal and time-consuming. In this paper, we
propose AutoML for Model Compression (AMC) which leverage reinforcement
learning to provide the model compression policy. This learning-based
compression policy outperforms conventional rule-based compression policy by
having higher compression ratio, better preserving the accuracy and freeing
human labor. Under 4x FLOPs reduction, we achieved 2.7% better accuracy than
the handcrafted model compression policy for VGG-16 on ImageNet. We applied
this automated, push-the-button compression pipeline to MobileNet and achieved
1.81x speedup of measured inference latency on an Android phone and 1.43x
speedup on the Titan XP GPU, with only 0.1% loss of ImageNet Top-1 accuracy.

本文介绍了使用强化学习算法构建的自动化神经网络模型压缩方法 ——AutoML for Model Compression（AMC），该方法在保持模型准确性的同时，相较于传统的基于规则的压缩方法，具有更高的压缩比，从而实现了在性能和精度之间的最优平衡，并将此方法成功应用于 MobileNet 优化。