The performance of transformers for time-series forecasting has improved
significantly. Recent architectures learn complex temporal patterns by
segmenting a time-series into patches and using the patches as tokens. The
patch size controls the ability of transformers to learn the temporal patterns
at different frequencies: shorter patches are effective for learning localized,
high-frequency patterns, whereas mining long-term seasonalities and trends
requires longer patches. Inspired by this observation, we propose a novel
framework, Multi-resolution Time-Series Transformer (MTST), which consists of a
multi-branch architecture for simultaneous modeling of diverse temporal
patterns at different resolutions. In contrast to many existing time-series
transformers, we employ relative positional encoding, which is better suited
for extracting periodic components at different scales. Extensive experiments
on several real-world datasets demonstrate the effectiveness of MTST in
comparison to state-of-the-art forecasting techniques.

提出了一种多分辨率时间序列变压器（MTST）框架，采用了多支架构来同时建模不同分辨率的多样化时间模式，通过相对位置编码提取不同尺度上的周期成分，与现有的时间序列变压器相比，在几个真实世界数据集上进行了广泛实验，证明了 MTST 在与最先进的预测技术的比较中的有效性。

多层次时间序列变换器用于长期预测

Multi-resolution Time-Series Transformer for Long-term Forecasting

Vision Transformers convert images to sequences by slicing them into patches.
The size of these patches controls a speed/accuracy tradeoff, with smaller
patches leading to higher accuracy at greater computational cost, but changing
the patch size typically requires retraining the model. In this paper, we
demonstrate that simply randomizing the patch size at training time leads to a
single set of weights that performs well across a wide range of patch sizes,
making it possible to tailor the model to different compute budgets at
deployment time. We extensively evaluate the resulting model, which we call
FlexiViT, on a wide range of tasks, including classification, image-text
retrieval, open-world detection, panoptic segmentation, and semantic
segmentation, concluding that it usually matches, and sometimes outperforms,
standard ViT models trained at a single patch size in an otherwise identical
setup. Hence, FlexiViT training is a simple drop-in improvement for ViT that
makes it easy to add compute-adaptive capabilities to most models relying on a
ViT backbone architecture. Code and pre-trained models are available at
this https URL

本文介绍一种叫做 FlexiViT 的方法，能够动态改变 ViT 模型输入的 patch size 以适应不同的计算预算，从而提高计算效率和精度。通过实验发现，FlexiViT 训练的模型在分类、图像文本检索、分割等多个任务上表现良好，易于应用于大多数基于 ViT 结构的计算任务。

FlexiViT：一模型多尺寸贴片处理

FlexiViT: One Model for All Patch Sizes

Semantic segmentation requires methods capable of learning high-level
features while dealing with large volume of data. Towards such goal,
Convolutional Networks can learn specific and adaptable features based on the
data. However, these networks are not capable of processing a whole remote
sensing image, given its huge size. To overcome such limitation, the image is
processed using fixed size patches. The definition of the input patch size is
usually performed empirically (evaluating several sizes) or imposed (by network
constraint). Both strategies suffer from drawbacks and could not lead to the
best patch size. To alleviate this problem, several works exploited
multi-context information by combining networks or layers. This process
increases the number of parameters resulting in a more difficult model to
train. In this work, we propose a novel technique to perform semantic
segmentation of remote sensing images that exploits a multi-context paradigm
without increasing the number of parameters while defining, in training time,
the best patch size. The main idea is to train a dilated network with distinct
patch sizes, allowing it to capture multi-context characteristics from
heterogeneous contexts. While processing these varying patches, the network
provides a score for each patch size, helping in the definition of the best
size for the current scenario. A systematic evaluation of the proposed
algorithm is conducted using four high-resolution remote sensing datasets with
very distinct properties. Our results show that the proposed algorithm provides
improvements in pixelwise classification accuracy when compared to
state-of-the-art methods.

本研究提出了一种利用多层感知域适用于遥感图像的语义分割的新技术，通过训练一个具有不同感知域的扩张网络，从多种异质性环境下获取信息，从训练样本中定义最佳输入补丁大小，显著提高了像素分类精度。