Several recent works have adapted Masked Autoencoders (MAEs) for learning
general-purpose audio representations. However, they do not address two key
aspects of modelling multi-domain audio data: (i) real-world audio tasks
consist of a combination of local+global contexts, and (ii) real-world audio
signals are complex compositions of several acoustic elements with different
time-frequency characteristics. To address these concerns, this work proposes a
Multi-Window Masked Autoencoder (MW-MAE) fitted with a novel Multi-Window
Multi-Head Attention module that can capture information at multiple local and
global contexts in every decoder transformer block through attention heads of
several distinct local and global windows. Empirical results on ten downstream
audio tasks show that MW-MAEs consistently outperform standard MAEs in overall
performance and learn better general-purpose audio representations, as well as
demonstrate considerably better scaling characteristics. Exploratory analyses
of the learned representations reveals that MW-MAE encoders learn attention
heads with more distinct entropies compared to those learned by MAEs, while
attention heads across the different transformer blocks in MW-MAE decoders
learn correlated feature representations, enabling each block to independently
capture local and global information, leading to a decoupled feature hierarchy.
Code for feature extraction and downstream experiments along with pre-trained
weights can be found at this https URL

本篇研究提出了一种使用多窗口的遮盖自编码器和多头注意力机制进行多领域音频数据建模的方法，并对比了标准遮盖自编码器的性能效果和特征获取能力。实验结果表明，多窗口的遮盖自编码器在处理多领域音频数据时具有更好的性能和特征提取能力，并且获得了更加独立的特征分层。

多窗口关注遮蔽自编码器更好的音频学习者

Masked Autoencoders with Multi-Window Attention Are Better Audio  Learners

Visual tempo characterizes the dynamics and the temporal scale of an action.
Modeling such visual tempos of different actions facilitates their recognition.
Previous works often capture the visual tempo through sampling raw videos at
multiple rates and constructing an input-level frame pyramid, which usually
requires a costly multi-branch network to handle. In this work we propose a
generic Temporal Pyramid Network (TPN) at the feature-level, which can be
flexibly integrated into 2D or 3D backbone networks in a plug-and-play manner.
Two essential components of TPN, the source of features and the fusion of
features, form a feature hierarchy for the backbone so that it can capture
action instances at various tempos. TPN also shows consistent improvements over
other challenging baselines on several action recognition datasets.
Specifically, when equipped with TPN, the 3D ResNet-50 with dense sampling
obtains a 2% gain on the validation set of Kinetics-400. A further analysis
also reveals that TPN gains most of its improvements on action classes that
have large variances in their visual tempos, validating the effectiveness of
TPN.

提出了一种特征层面上的通用时间金字塔网络（Temporal Pyramid Network），可以在 2D 或 3D 骨干网络中灵活地进行插拔式集成，通过构建功能层次结构来捕捉不同节奏的动作实例，并在几个行动识别数据集上显示出比其他具有挑战性的基线更为一致的改进。

动作识别的时间金字塔网络

Temporal Pyramid Network for Action Recognition

Interest point descriptors have fueled progress on almost every problem in
computer vision. Recent advances in deep neural networks have enabled
task-specific learned descriptors that outperform hand-crafted descriptors on
many problems. We demonstrate that commonly used metric learning approaches do
not optimally leverage the feature hierarchies learned in a Convolutional
Neural Network (CNN), especially when applied to the task of geometric feature
matching. While a metric loss applied to the deepest layer of a CNN, is often
expected to yield ideal features irrespective of the task, in fact the growing
receptive field as well as striding effects cause shallower features to be
better at high precision matching tasks. We leverage this insight together with
explicit supervision at multiple levels of the feature hierarchy for better
regularization, to learn more effective descriptors in the context of geometric
matching tasks. Further, we propose to use activation maps at different layers
of a CNN, as an effective and principled replacement for the multi-resolution
image pyramids often used for matching tasks. We propose concrete CNN
architectures employing these ideas, and evaluate them on multiple datasets for
2D and 3D geometric matching as well as optical flow, demonstrating
state-of-the-art results and generalization across datasets.

该研究提出了使用深度学习的 CNN 网络以及不同层次的特征图来进行几何匹配的任务，并相较于先前的手动特征点描述方法，展示了更佳的匹配结果。

层次结构度量学习及其在二维和三维几何对应中的匹配

Hierarchical Metric Learning and Matching for 2D and 3D Geometric  Correspondences

Visual recognition requires rich representations that span levels from low to
high, scales from small to large, and resolutions from fine to coarse. Even
with the depth of features in a convolutional network, a layer in isolation is
not enough: compounding and aggregating these representations improves
inference of what and where. Architectural efforts are exploring many
dimensions for network backbones, designing deeper or wider architectures, but
how to best aggregate layers and blocks across a network deserves further
attention. Although skip connections have been incorporated to combine layers,
these connections have been "shallow" themselves, and only fuse by simple,
one-step operations. We augment standard architectures with deeper aggregation
to better fuse information across layers. Our deep layer aggregation structures
iteratively and hierarchically merge the feature hierarchy to make networks
with better accuracy and fewer parameters. Experiments across architectures and
tasks show that deep layer aggregation improves recognition and resolution
compared to existing branching and merging schemes. The code is at
this https URL

通过进行深度聚合来更好地融合图像识别中层间的信息，并对网络体系结构的各个方面进行了实验研究，发现深度聚合可以提高识别效果并减少网络参数。