We present the first openly available multimodal metaphor annotated corpus.
The corpus consists of videos including audio and subtitles that have been
annotated by experts. Furthermore, we present a method for detecting metaphors
in the new dataset based on the textual content of the videos. The method
achieves a high F1-score (62\%) for metaphorical labels. We also experiment
with other modalities and multimodal methods; however, these methods did not
out-perform the text-based model. In our error analysis, we do identify that
there are cases where video could help in disambiguating metaphors, however,
the visual cues are too subtle for our model to capture. The data is available
on Zenodo.

本研究提供了第一个公开可用的多模式隐喻标注语料库，并尝试了多种模态和多模式方法，发现基于文本内容的模型能够最有效地检测隐喻，并且在误差分析中发现视觉线索对隐喻歧义消歧有帮助，但太微妙难以捕捉，并将数据上传到 Zenodo 平台。

环环相扣：一个视频多模态隐喻检测的语料库和方法

Ring That Bell: A Corpus and Method for Multimodal Metaphor Detection in Videos

Tables are an important form of structured data for both human and machine
readers alike, providing answers to questions that cannot, or cannot easily, be
found in texts. Recent work has designed special models and training paradigms
for table-related tasks such as table-based question answering and table
retrieval. Though effective, they add complexity in both modeling and data
acquisition compared to generic text solutions and obscure which elements are
truly beneficial. In this work, we focus on the task of table retrieval, and
ask: "is table-specific model design necessary for table retrieval, or can a
simpler text-based model be effectively used to achieve a similar result?"
First, we perform an analysis on a table-based portion of the Natural Questions
dataset (NQ-table), and find that structure plays a negligible role in more
than 70% of the cases. Based on this, we experiment with a general Dense
Passage Retriever (DPR) based on text and a specialized Dense Table Retriever
(DTR) that uses table-specific model designs. We find that DPR performs well
without any table-specific design and training, and even achieves superior
results compared to DTR when fine-tuned on properly linearized tables. We then
experiment with three modules to explicitly encode table structures, namely
auxiliary row/column embeddings, hard attention masks, and soft relation-based
attention biases. However, none of these yielded significant improvements,
suggesting that table-specific model design may not be necessary for table
retrieval.

本研究旨在探究是否需要使用特别设计的表格型模型进行表格信息检索，结果表明在大部分情况下使用基于文本的模型，如 Dense Passage Retriever (DPR) 即可达到相似或更优秀的结果，无需特别设计。通过实验，辅助行 / 列嵌入、硬性注意力蒙版和软关系注意力偏差等三个模块也未能产生显著提高效果，表明使用特别设计的模型并非必要。