Using images acquired by different satellite sensors has shown to improve
classification performance in the framework of crop mapping from satellite
image time series (SITS). Existing state-of-the-art architectures use
self-attention mechanisms to process the temporal dimension and convolutions
for the spatial dimension of SITS. Motivated by the success of purely
attention-based architectures in crop mapping from single-modal SITS, we
introduce several multi-modal multi-temporal transformer-based architectures.
Specifically, we investigate the effectiveness of Early Fusion, Cross Attention
Fusion and Synchronized Class Token Fusion within the Temporo-Spatial Vision
Transformer (TSViT). Experimental results demonstrate significant improvements
over state-of-the-art architectures with both convolutional and self-attention
components.

通过使用来自不同卫星传感器获取的图像，已经证实在卫星图像时间序列（SITS）的作物分布图框架中，可以提高分类性能。现有的最新架构使用自注意机制处理时间维度和卷积处理空间维度。受到单模态 SITS 作物分布图中纯注意力架构的成功启发，我们引入了几种多模态多时序变换器架构。具体来说，我们研究了在时间空间视觉变换器（TSViT）中早期融合、交叉注意融合和同步类标记融合的有效性。实验结果表明，相较于具有卷积和自注意组件的最新架构，我们的架构显著改善了性能。