Deep neural networks (DNNs) are frequently employed in a variety of computer
vision applications. Nowadays, an emerging trend in the current video
distribution system is to take advantage of DNN's overfitting properties to
perform video resolution upscaling. By splitting videos into chunks and
applying a super-resolution (SR) model to overfit each chunk, this scheme of SR
models plus video chunks is able to replace traditional video transmission to
enhance video quality and transmission efficiency. However, many models and
chunks are needed to guarantee high performance, which leads to tremendous
overhead on model switching and memory footprints at the user end. To resolve
such problems, we propose a Dynamic Deep neural network assisted by a
Content-Aware data processing pipeline to reduce the model number down to one
(Dy-DCA), which helps promote performance while conserving computational
resources. Additionally, to achieve real acceleration on the user end, we
designed a framework that optimizes dynamic features (e.g., dynamic shapes,
sizes, and control flow) in Dy-DCA to enable a series of compilation
optimizations, including fused code generation, static execution planning, etc.
By employing such techniques, our method achieves better PSNR and real-time
performance (33 FPS) on an off-the-shelf mobile phone. Meanwhile, assisted by
our compilation optimization, we achieve a 1.7$\times$ speedup while saving up
to 1.61$\times$ memory consumption. Code available in
this https URL

通过动态深度神经网络辅助的内容感知数据处理管道，我们提出了一种减少模型数量的方法，以提高性能并节省计算资源，并通过编译优化实现更好的 PSNR 和实时性能，在现成的手机上实现了 1.7 倍的加速，节省了高达 1.61 倍的内存消耗。

基于动态算法和编译器协同设计的设备端超分辨率中的数据过拟合

Data Overfitting for On-Device Super-Resolution with Dynamic Algorithm  and Compiler Co-Design

We present RGB-D-Fusion, a multi-modal conditional denoising diffusion
probabilistic model to generate high resolution depth maps from low-resolution
monocular RGB images of humanoid subjects. RGB-D-Fusion first generates a
low-resolution depth map using an image conditioned denoising diffusion
probabilistic model and then upsamples the depth map using a second denoising
diffusion probabilistic model conditioned on a low-resolution RGB-D image. We
further introduce a novel augmentation technique, depth noise augmentation, to
increase the robustness of our super-resolution model.

RGB-D-Fusion 模型使用条件去噪扩散概率模型从人类主体的低分辨率单眼 RGB 图像生成高分辨率深度图，并引入深度噪声增强技术以提升超分辨率模型的鲁棒性。