Existing learned video compression models employ flow net or deformable
convolutional networks (DCN) to estimate motion information. However, the
limited receptive fields of flow net and DCN inherently direct their
attentiveness towards the local contexts. Global contexts, such as large-scale
motions and global correlations among frames are ignored, presenting a
significant bottleneck for capturing accurate motions. To address this issue,
we propose a joint local and global motion compensation module (LGMC) for
leaned video coding. More specifically, we adopt flow net for local motion
compensation. To capture global context, we employ the cross attention in
feature domain for motion compensation. In addition, to avoid the quadratic
complexity of vanilla cross attention, we divide the softmax operations in
attention into two independent softmax operations, leading to linear
complexity. To validate the effectiveness of our proposed LGMC, we integrate it
with DCVC-TCM and obtain learned video compression with joint local and global
motion compensation (LVC-LGMC). Extensive experiments demonstrate that our
LVC-LGMC has significant rate-distortion performance improvements over baseline
DCVC-TCM.

通过引入联合的局部和全局运动补偿模块（LGMC），本研究提出了一种学习视频编码的方法，结合了局部和全局运动补偿，有效提高了速率失真性能。

LVC-LGMC：学习式视频压缩的联合局部与全局运动补偿

LVC-LGMC: Joint Local and Global Motion Compensation for Learned Video  Compression

This work focuses on mitigating two limitations in the joint learning of
local feature detectors and descriptors. First, the ability to estimate the
local shape (scale, orientation, etc.) of feature points is often neglected
during dense feature extraction, while the shape-awareness is crucial to
acquire stronger geometric invariance. Second, the localization accuracy of
detected keypoints is not sufficient to reliably recover camera geometry, which
has become the bottleneck in tasks such as 3D reconstruction. In this paper, we
present ASLFeat, with three light-weight yet effective modifications to
mitigate above issues. First, we resort to deformable convolutional networks to
densely estimate and apply local transformation. Second, we take advantage of
the inherent feature hierarchy to restore spatial resolution and low-level
details for accurate keypoint localization. Finally, we use a peakiness
measurement to relate feature responses and derive more indicative detection
scores. The effect of each modification is thoroughly studied, and the
evaluation is extensively conducted across a variety of practical scenarios.
State-of-the-art results are reported that demonstrate the superiority of our
methods.

ASLFeat 是一种新的本地特征检测器和描述符结合的方法，使用可变形卷积网络估计本地特征点的形状，从而实现更强的几何不变性。此外，ASLFeat 使用层次结构以恢复空间分辨率和局部细节，并通过尖锐度测量来推导最有指示性的检测分数，以进一步提高检测精度和可靠性，并可在各种实际场景中获得最新的研究成果。

ASLFeat: 学习准确形状和定位的本地特征

ASLFeat: Learning Local Features of Accurate Shape and Localization

The superior performance of Deformable Convolutional Networks arises from its
ability to adapt to the geometric variations of objects. Through an examination
of its adaptive behavior, we observe that while the spatial support for its
neural features conforms more closely than regular ConvNets to object
structure, this support may nevertheless extend well beyond the region of
interest, causing features to be influenced by irrelevant image content. To
address this problem, we present a reformulation of Deformable ConvNets that
improves its ability to focus on pertinent image regions, through increased
modeling power and stronger training. The modeling power is enhanced through a
more comprehensive integration of deformable convolution within the network,
and by introducing a modulation mechanism that expands the scope of deformation
modeling. To effectively harness this enriched modeling capability, we guide
network training via a proposed feature mimicking scheme that helps the network
to learn features that reflect the object focus and classification power of
R-CNN features. With the proposed contributions, this new version of Deformable
ConvNets yields significant performance gains over the original model and
produces leading results on the COCO benchmark for object detection and
instance segmentation.

通过增强 Deformable Convolutional Networks 的变形卷积能力、引入调制机制和特征模仿方案，提高了其在目标检测和实例分割任务中的性能表现。