Existing approaches for learning representations of time-series keep the
temporal arrangement of the time-steps intact with the presumption that the
original order is the most optimal for learning. However, non-adjacent sections
of real-world time-series may have strong dependencies. Accordingly we raise
the question: Is there an alternative arrangement for time-series which could
enable more effective representation learning? To address this, we propose a
simple plug-and-play mechanism called Segment, Shuffle, and Stitch (S3)
designed to improve time-series representation learning of existing models. S3
works by creating non-overlapping segments from the original sequence and
shuffling them in a learned manner that is the most optimal for the task at
hand. It then re-attaches the shuffled segments back together and performs a
learned weighted sum with the original input to capture both the newly shuffled
sequence along with the original sequence. S3 is modular and can be stacked to
create various degrees of granularity, and can be added to many forms of neural
architectures including CNNs or Transformers with negligible computation
overhead. Through extensive experiments on several datasets and
state-of-the-art baselines, we show that incorporating S3 results in
significant improvements for the tasks of time-series classification and
forecasting, improving performance on certain datasets by up to 68\%. We also
show that S3 makes the learning more stable with a smoother training loss curve
and loss landscape compared to the original baseline. The code is available at
this https URL .

现有的学习时间序列的方法保持时间步骤的时间顺序，假设原始顺序对学习来说是最优的。然而，现实世界的时间序列的非相邻部分可能具有强烈的依赖关系。为了解决这个问题，我们提出了一种名为 Segment, Shuffle, and Stitch（S3）的简单即插即用机制，旨在改进现有模型的时间序列表示学习。S3 通过从原始序列中创建非重叠的片段并以一种在任务中最优的方式对它们进行洗牌，然后将洗牌后的片段重新连接并执行与原始输入的学习加权求和，以捕获新的洗牌序列和原始序列。S3 是模块化的，可以堆叠以创建不同程度的粒度，并可添加到许多形式的神经架构，包括卷积神经网络（CNN）或 Transformer，计算开销可忽略。通过对多个数据集和现有基线进行大量实验证明，引入 S3 可以显著提高时间序列分类和预测任务的性能，在某些数据集上的性能提高高达 68％。我们还展示了 S3 相比于原始基线使学习更稳定，具有较平滑的训练损失曲线和损失曲面。代码可在此 https URL 中获得。

分割、洗牌和拼接：改进时间序列表示的简单机制

Segment, Shuffle, and Stitch: A Simple Mechanism for Improving  Time-Series Representations

Vision Transformer has shown great visual representation power in substantial
vision tasks such as recognition and detection, and thus been attracting
fast-growing efforts on manually designing more effective architectures. In
this paper, we propose to use neural architecture search to automate this
process, by searching not only the architecture but also the search space. The
central idea is to gradually evolve different search dimensions guided by their
E-T Error computed using a weight-sharing supernet. Moreover, we provide design
guidelines of general vision transformers with extensive analysis according to
the space searching process, which could promote the understanding of vision
transformer. Remarkably, the searched models, named S3 (short for Searching the
Search Space), from the searched space achieve superior performance to recently
proposed models, such as Swin, DeiT and ViT, when evaluated on ImageNet. The
effectiveness of S3 is also illustrated on object detection, semantic
segmentation and visual question answering, demonstrating its generality to
downstream vision and vision-language tasks. Code and models will be available
at this https URL

本文介绍了使用神经架构搜索来自动化提高 Vision Transformer 架构和搜索空间的过程，以及设计指南和分析。通过使用 E-T Error 来逐步演化不同的搜索维度。此外，从所搜索的空间中得到的模型，名为 S3，评估在 ImageNet 上表现比最近提出的模型（如 Swin、DeiT 和 ViT）更出色，并在目标检测、语义分割和视觉问答等任务中展示了其普适性。

搜索视觉 Transformer 的搜索空间

Searching the Search Space of Vision Transformer

Multimodal IR, spanning text corpus, knowledge graph and images, called
outside knowledge visual question answering (OKVQA), is of much recent
interest. However, the popular data set has serious limitations. A surprisingly
large fraction of queries do not assess the ability to integrate cross-modal
information. Instead, some are independent of the image, some depend on
speculation, some require OCR or are otherwise answerable from the image alone.
To add to the above limitations, frequency-based guessing is very effective
because of (unintended) widespread answer overlaps between the train and test
folds. Overall, it is hard to determine when state-of-the-art systems exploit
these weaknesses rather than really infer the answers, because they are opaque
and their 'reasoning' process is uninterpretable. An equally important
limitation is that the dataset is designed for the quantitative assessment only
of the end-to-end answer retrieval task, with no provision for assessing the
correct(semantic) interpretation of the input query. In response, we identify a
key structural idiom in OKVQA ,viz., S3 (select, substitute and search), and
build a new data set and challenge around it. Specifically, the questioner
identifies an entity in the image and asks a question involving that entity
which can be answered only by consulting a knowledge graph or corpus passage
mentioning the entity. Our challenge consists of (i)OKVQAS3, a subset of OKVQA
annotated based on the structural idiom and (ii)S3VQA, a new dataset built from
scratch. We also present a neural but structurally transparent OKVQA system,
S3, that explicitly addresses our challenge dataset, and outperforms recent
competitive baselines.

该研究针对现有数据集中存在的限制，建立了一个新的基于结构模式的跨模态问答数据集和挑战，同时提出了一个基于神经网络的透明的 OKVQA 系统 S3，该系统在新数据集上实现了比较好的表现。