Multimodal aspect-based sentiment analysis (MABSA) aims to understand
opinions in a granular manner, advancing human-computer interaction and other
fields. Traditionally, MABSA methods use a joint prediction approach to
identify aspects and sentiments simultaneously. However, we argue that joint
models are not always superior. Our analysis shows that joint models struggle
to align relevant text tokens with image patches, leading to misalignment and
ineffective image utilization.
In contrast, a pipeline framework first identifies aspects through MATE
(Multimodal Aspect Term Extraction) and then aligns these aspects with image
patches for sentiment classification (MASC: Multimodal Aspect-Oriented
Sentiment Classification). This method is better suited for multimodal
scenarios where effective image use is crucial. We present three key
observations: (a) MATE and MASC have different feature requirements, with MATE
focusing on token-level features and MASC on sequence-level features; (b) the
aspect identified by MATE is crucial for effective image utilization; and (c)
images play a trivial role in previous MABSA methods due to high noise.
Based on these observations, we propose a pipeline framework that first
predicts the aspect and then uses translation-based alignment (TBA) to enhance
multimodal semantic consistency for better image utilization. Our method
achieves state-of-the-art (SOTA) performance on widely used MABSA datasets
Twitter-15 and Twitter-17. This demonstrates the effectiveness of the pipeline
approach and its potential to provide valuable insights for future MABSA
research.
For reproducibility, the code and checkpoint will be released.

多模态基于方面的情感分析 (MABSA) 研究了一个管道框架，其中首先通过 MATE 多模态方面术语抽取识别方面，然后将这些方面与图像补丁对齐用于情感分类 (MASC)，并且该方法在 Twitter-15 和 Twitter-17 数据集上取得了最先进的性能表现。

通过流水线预测和基于翻译的对齐，增强多模态情感分析

PTA: Enhancing Multimodal Sentiment Analysis through Pipelined  Prediction and Translation-based Alignment

This work presents a sparse-attention Transformer architecture for modeling
documents that contain large tables. Tables are ubiquitous on the web, and are
rich in information. However, more than 20% of relational tables on the web
have 20 or more rows (Cafarella et al., 2008), and these large tables present a
challenge for current Transformer models, which are typically limited to 512
tokens. Here we propose MATE, a novel Transformer architecture designed to
model the structure of web tables. MATE uses sparse attention in a way that
allows heads to efficiently attend to either rows or columns in a table. This
architecture scales linearly with respect to speed and memory, and can handle
documents containing more than 8000 tokens with current accelerators. MATE also
has a more appropriate inductive bias for tabular data, and sets a new
state-of-the-art for three table reasoning datasets. For HybridQA (Chen et al.,
2020b), a dataset that involves large documents containing tables, we improve
the best prior result by 19 points.

使用稀疏注意力 Transformer 建构的 MATE 结构，可用于处理包含大量表格的网页文档。MATE 能够高效地处理表格的行列信息，并对涵盖超过 8000 个单元格的文档进行建模，同时在处理包含表格的大型文档时也表现出优越性。我们在三个涉及表格的推理数据集上创造了最新的最佳结果，其中包括了大型文档和表格的 HybridQA 数据集，我们的表现提高了 19 分。