Detecting and magnifying imperceptible high-frequency motions in real-world
scenarios has substantial implications for industrial and medical applications.
These motions are characterized by small amplitudes and high frequencies.
Traditional motion magnification methods rely on costly high-speed cameras or
active light sources, which limit the scope of their applications. In this
work, we propose a dual-camera system consisting of an event camera and a
conventional RGB camera for video motion magnification, containing
temporally-dense information from the event stream and spatially-dense data
from the RGB images. This innovative combination enables a broad and
cost-effective amplification of high-frequency motions. By revisiting the
physical camera model, we observe that estimating motion direction and
magnitude necessitates the integration of event streams with additional image
features. On this basis, we propose a novel deep network for event-based video
motion magnification that addresses two primary challenges: firstly, the high
frequency of motion induces a large number of interpolated frames (up to 80),
which our network mitigates with a Second-order Recurrent Propagation module
for better handling of long-term frame interpolations; and secondly, magnifying
subtle motions is sensitive to noise, which we address by utilizing a temporal
filter to amplify motion at specific frequencies and reduce noise impact. We
demonstrate the effectiveness and accuracy of our dual-camera system and
network through extensive experiments in magnifying small-amplitude,
high-frequency motions, offering a cost-effective and flexible solution for
motion detection and magnification.

提出了一种由事件相机和传统 RGB 相机组成的双摄像头系统，结合了事件流的时间密集信息和 RGB 图像的空间密集数据，以实现对高频运动的广泛和经济有效的放大，通过深度网络解决了高频运动和微小运动放大的问题。