Remote sensing images present unique challenges to image analysis due to the
extensive geographic coverage, hardware limitations, and misaligned multi-scale
images. This paper revisits the classical multi-scale representation learning
problem but under the general framework of self-supervised learning for remote
sensing image understanding. We present Cross-Scale MAE, a self-supervised
model built upon the Masked Auto-Encoder (MAE).During pre-training, Cross-Scale
MAE employs scale augmentation techniques and enforces cross-scale consistency
constraints through both contrastive and generative losses to ensure consistent
and meaningful representations well-suited for a wide range of downstream
tasks. Further, our implementation leverages the xFormers library to accelerate
network pre-training on a single GPU while maintaining the quality of learned
representations. Experimental evaluations demonstrate that Cross-Scale MAE
exhibits superior performance compared to standard MAE and other
state-of-the-art remote sensing MAE methods.

远程感知图像理解中，该研究通过自监督学习的通用框架重新探讨了经典的多尺度表示学习问题，提出了一种基于掩码自编码器（MAE）的 Cross-Scale MAE 自监督模型，通过尺度增强技术以及对比损失和生成损失的交叉尺度一致性约束来确保一致且有意义的表示，进一步利用 xFormers 库在单个 GPU 上加速网络预训练并保持所学表示的质量，实验证明 Cross-Scale MAE 相较于标准 MAE 和其他最先进的远程感知 MAE 方法具有优越的性能。

跨尺度 MAE: 遥感中多尺度利用的故事

Cross-Scale MAE: A Tale of Multi-Scale Exploitation in Remote Sensing

Diffusion models (DMs) have shown great potential for high-quality image
synthesis. However, when it comes to producing images with complex scenes, how
to properly describe both image global structures and object details remains a
challenging task. In this paper, we present Frido, a Feature Pyramid Diffusion
model performing a multi-scale coarse-to-fine denoising process for image
synthesis. Our model decomposes an input image into scale-dependent vector
quantized features, followed by a coarse-to-fine gating for producing image
output. During the above multi-scale representation learning stage, additional
input conditions like text, scene graph, or image layout can be further
exploited. Thus, Frido can be also applied for conditional or cross-modality
image synthesis. We conduct extensive experiments over various unconditioned
and conditional image generation tasks, ranging from text-to-image synthesis,
layout-to-image, scene-graph-to-image, to label-to-image. More specifically, we
achieved state-of-the-art FID scores on five benchmarks, namely layout-to-image
on COCO and OpenImages, scene-graph-to-image on COCO and Visual Genome, and
label-to-image on COCO. Code is available at
this https URL.

本文介绍一种特征金字塔扩散模型 Frido，通过多尺度的粗到细的去噪过程将输入图像分解为尺度依赖型向量量化特征，并在图像输出时进行粗到细的门控操作以产生图像输出。通过在多尺度表示学习阶段利用其他输入条件如文本、场景图或图像布局，Frido 还可以用于条件或跨模态图像合成。作者在文中对各种无条件和有条件的图像合成任务进行了广泛的实验，包括从文本到图像的合成、布局到图像、场景图到图像以及标签到图像等，取得了五个基准测试中的最新的 FID 分数，包括 COCO 和 OpenImages 上的布局到图像、COCO 和 Visual Genome 上的场景图到图像以及 COCO 上的标签到图像。

Frido: 复杂场景图像综合的特征金字塔扩散

Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis

In sequence to sequence learning, the self-attention mechanism proves to be
highly effective, and achieves significant improvements in many tasks. However,
the self-attention mechanism is not without its own flaws. Although
self-attention can model extremely long dependencies, the attention in deep
layers tends to overconcentrate on a single token, leading to insufficient use
of local information and difficultly in representing long sequences. In this
work, we explore parallel multi-scale representation learning on sequence data,
striving to capture both long-range and short-range language structures. To
this end, we propose the Parallel MUlti-Scale attEntion (MUSE) and MUSE-simple.
MUSE-simple contains the basic idea of parallel multi-scale sequence
representation learning, and it encodes the sequence in parallel, in terms of
different scales with the help from self-attention, and pointwise
transformation. MUSE builds on MUSE-simple and explores combining convolution
and self-attention for learning sequence representations from more different
scales. We focus on machine translation and the proposed approach achieves
substantial performance improvements over Transformer, especially on long
sequences. More importantly, we find that although conceptually simple, its
success in practice requires intricate considerations, and the multi-scale
attention must build on unified semantic space. Under common setting, the
proposed model achieves substantial performance and outperforms all previous
models on three main machine translation tasks. In addition, MUSE has potential
for accelerating inference due to its parallelism. Code will be available at
this https URL

本研究旨在探索并提出一种并行多尺度表示学习方法，利用自注意力和点对点转换，对序列数据进行多尺度编码和建模，实现长距离和短距离结构中的机器翻译任务，该方法在三个主要机器翻译任务中表现优异，并具有加速推理的潜力。