In this paper, we present RStab, a novel framework for video stabilization
that integrates 3D multi-frame fusion through volume rendering. Departing from
conventional methods, we introduce a 3D multi-frame perspective to generate
stabilized images, addressing the challenge of full-frame generation while
preserving structure. The core of our approach lies in Stabilized Rendering
(SR), a volume rendering module, which extends beyond the image fusion by
incorporating feature fusion. The core of our RStab framework lies in
Stabilized Rendering (SR), a volume rendering module, fusing multi-frame
information in 3D space. Specifically, SR involves warping features and colors
from multiple frames by projection, fusing them into descriptors to render the
stabilized image. However, the precision of warped information depends on the
projection accuracy, a factor significantly influenced by dynamic regions. In
response, we introduce the Adaptive Ray Range (ARR) module to integrate depth
priors, adaptively defining the sampling range for the projection process.
Additionally, we propose Color Correction (CC) assisting geometric constraints
with optical flow for accurate color aggregation. Thanks to the three modules,
our RStab demonstrates superior performance compared with previous stabilizers
in the field of view (FOV), image quality, and video stability across various
datasets.

本文介绍了一种名为 RStab 的视频稳定化新框架，它通过体积渲染集成了三维多帧融合。我们引入三维多帧视角来生成稳定图像，解决了全帧生成和结构保留的挑战。该框架核心在于稳定渲染（SR）模块，在图像融合的基础上还融合了特征。通过投影将多帧的特征和颜色进行变形和融合，生成稳定图像。此外，我们引入自适应射线范围（ARR）模块来集成深度先验，自适应定义投影过程的采样范围。同时，我们提出了色彩校正（CC）辅助几何约束和光流以实现准确的颜色聚合。多亏了这三个模块，我们的 RStab 在视野范围、图像质量和视频稳定性等各种数据集上展现出卓越的性能。