Aerial surveillance requires high spatio-temporal resolution (HSTR) video for
more accurate detection and tracking of objects. This is especially true for
wide-area surveillance (WAS), where the surveyed region is large and the
objects of interest are small. This paper proposes a dual camera system for the
generation of HSTR video using reference-based super-resolution (RefSR). One
camera captures high spatial resolution low frame rate (HSLF) video while the
other captures low spatial resolution high frame rate (LSHF) video
simultaneously for the same scene. A novel deep learning architecture is
proposed to fuse HSLF and LSHF video feeds and synthesize HSTR video frames at
the output. The proposed model combines optical flow estimation and
(channel-wise and spatial) attention mechanisms to capture the fine motion and
intricate dependencies between frames of the two video feeds. Simulations show
that the proposed model provides significant improvement over existing
reference-based SR techniques in terms of PSNR and SSIM metrics. The method
also exhibits sufficient frames per second (FPS) for WAS when deployed on a
power-constrained drone equipped with dual cameras.

本文提出了一种用于航空监视的双摄像头系统，通过基于参考的超分辨率生成高空间 - 时间分辨率视频帧，并结合光流估计和注意机制，以捕捉两个视频源帧之间的细微运动和复杂依赖关系。通过模拟实验，证明该模型在 PSNR 和 SSIM 度量方面相较现有的基于参考的超分辨率技术取得显著改进，并在功耗受限的无人机上部署时表现出足够的帧率适用于广域航空监视。

HSTR-Net: 基于参考图像的双摄像头航空监视视频超分辨率

HSTR-Net: Reference Based Video Super-resolution for Aerial Surveillance  with Dual Cameras

The rapid emergence of airborne platforms and imaging sensors are enabling
new forms of aerial surveillance due to their unprecedented advantages in
scale, mobility, deployment and covert observation capabilities. This paper
provides a comprehensive overview of human-centric aerial surveillance tasks
from a computer vision and pattern recognition perspective. It aims to provide
readers with an in-depth systematic review and technical analysis of the
current state of aerial surveillance tasks using drones, UAVs and other
airborne platforms. The main object of interest is humans, where single or
multiple subjects are to be detected, identified, tracked, re-identified and
have their behavior analyzed. More specifically, for each of these four tasks,
we first discuss unique challenges in performing these tasks in an aerial
setting compared to a ground-based setting. We then review and analyze the
aerial datasets publicly available for each task, and delve deep into the
approaches in the aerial literature and investigate how they presently address
the aerial challenges. We conclude the paper with discussion on the missing
gaps and open research questions to inform future research avenues.

本文旨在从计算机视觉和模式识别角度全面介绍以人为中心的空中监视任务，主要关注无人机、无人航空器和其他航空平台的应用，探讨其在人类检测、识别、跟踪、行为分析方面的挑战及应对方法，并提出未来研究方向。