Data augmentation is a prevalent technique for improving performance in
various machine learning applications. We propose SeqAug, a modality-agnostic
augmentation method that is tailored towards sequences of extracted features.
The core idea of SeqAug is to augment the sequence by resampling from the
underlying feature distribution. Resampling is performed by randomly selecting
feature dimensions and permuting them along the temporal axis. Experiments on
CMU-MOSEI verify that SeqAug is modality agnostic; it can be successfully
applied to a single modality or multiple modalities. We further verify its
compatibility with both recurrent and transformer architectures, and also
demonstrate comparable to state-of-the-art results.

提出了一种名为 SeqAug 的模态不可知序列增强方法，它可以通过重新抽样来增强时间轴上的特征维度，可应用于单模态或多模态神经网络，并与循环和转换器体系结构兼容。实验证明，SeqAug 在语音听觉情感识别方面具有可比性的性能。

SeqAug: 一种基于序列特征重采样的通用增强方法

SeqAug: Sequential Feature Resampling as a modality agnostic  augmentation method

Motion forecasting for autonomous driving is a challenging task because
complex driving scenarios result in a heterogeneous mix of static and dynamic
inputs. It is an open problem how best to represent and fuse information about
road geometry, lane connectivity, time-varying traffic light state, and history
of a dynamic set of agents and their interactions into an effective encoding.
To model this diverse set of input features, many approaches proposed to design
an equally complex system with a diverse set of modality specific modules. This
results in systems that are difficult to scale, extend, or tune in rigorous
ways to trade off quality and efficiency. In this paper, we present Wayformer,
a family of attention based architectures for motion forecasting that are
simple and homogeneous. Wayformer offers a compact model description consisting
of an attention based scene encoder and a decoder. In the scene encoder we
study the choice of early, late and hierarchical fusion of the input
modalities. For each fusion type we explore strategies to tradeoff efficiency
and quality via factorized attention or latent query attention. We show that
early fusion, despite its simplicity of construction, is not only modality
agnostic but also achieves state-of-the-art results on both Waymo Open
MotionDataset (WOMD) and Argoverse leaderboards, demonstrating the
effectiveness of our design philosophy

本文提出了一种基于注意力机制的动作预测体系结构，并探讨了不同融合方法的有效性。结果表明，相比于其他模态融合方法，早期融合的模态无关架构在 Waymo Open MotionDataset (WOMD) 和 Argoverse 排行榜中都有着最优秀的表现。

Wayformer：基于简单高效注意力网络的运动预测

Wayformer: Motion Forecasting via Simple & Efficient Attention Networks

State-of-the-art encoder-decoder models (e.g. for machine translation (MT) or
speech recognition (ASR)) are constructed and trained end-to-end as an atomic
unit. No component of the model can be (re-)used without the others. We
describe LegoNN, a procedure for building encoder-decoder architectures with
decoder modules that can be reused across various MT and ASR tasks, without the
need for any fine-tuning. To achieve reusability, the interface between each
encoder and decoder modules is grounded to a sequence of marginal distributions
over a discrete vocabulary pre-defined by the model designer. We present two
approaches for ingesting these marginals; one is differentiable, allowing the
flow of gradients across the entire network, and the other is
gradient-isolating. To enable portability of decoder modules between MT tasks
for different source languages and across other tasks like ASR, we introduce a
modality agnostic encoder which consists of a length control mechanism to
dynamically adapt encoders' output lengths in order to match the expected input
length range of pre-trained decoders. We present several experiments to
demonstrate the effectiveness of LegoNN models: a trained language generation
LegoNN decoder module from German-English (De-En) MT task can be reused with no
fine-tuning for the Europarl English ASR and the Romanian-English (Ro-En) MT
tasks to match or beat respective baseline models. When fine-tuned towards the
target task for few thousand updates, our LegoNN models improved the Ro-En MT
task by 1.5 BLEU points, and achieved 12.5% relative WER reduction for the
Europarl ASR task. Furthermore, to show its extensibility, we compose a LegoNN
ASR model from three modules -- each has been learned within different
end-to-end trained models on three different datasets -- boosting the WER
reduction to 19.5%.

描述了 LegoNN 过程，它通过重新使用 decoder 模块来构建 encoder-decoder 结构，从而实现在各种机器翻译和语音识别任务中的重复使用，同时引入了一种模态不可知编码器来提高其可移植性。实验证明 LegoNN 模型的有效性。