Multimodal Large Language Models (MLLMs) have demonstrated proficiency in
handling a variety of visual-language tasks. However, current MLLM benchmarks
are predominantly designed to evaluate reasoning based on static information
about a single image, and the ability of modern MLLMs to extrapolate from image
sequences, which is essential for understanding our ever-changing world, has
been less investigated. To address this challenge, this paper introduces
Mementos, a new benchmark designed to assess MLLMs' sequential image reasoning
abilities. Mementos features 4,761 diverse image sequences with varying
lengths. We also employ a GPT-4 assisted method to evaluate MLLM reasoning
performance. Through a careful evaluation of nine recent MLLMs on Mementos,
including GPT-4V and Gemini, we find that they struggle to accurately describe
dynamic information about given image sequences, often leading to
hallucinations/misrepresentations of objects and their corresponding behaviors.
Our quantitative analysis and case studies identify three key factors impacting
MLLMs' sequential image reasoning: the correlation between object and
behavioral hallucinations, the influence of cooccurring behaviors, and the
compounding impact of behavioral hallucinations. Our dataset is available at
this https URL

这篇论文介绍了 Mementos，这是一个旨在评估多模态大型语言模型（MLLMs）的顺序图像推理能力的新基准，通过评估九种最新的 MLMMs 对 Mementos 的表现，发现它们在准确描述给定图像序列的动态信息方面存在困难，经常导致物体及其相关行为的幻觉 / 误代，研究确定了影响 MLLMs 顺序图像推理的三个关键因素：物体和行为幻觉之间的相关性，共发生行为的影响以及行为幻觉的复合影响。

记念品：一种用于模态多模式大型语言模型推理图像序列的综合基准

Mementos: A Comprehensive Benchmark for Multimodal Large Language Model  Reasoning over Image Sequences

Event-based camera is a bio-inspired vision sensor that records intensity
changes (called event) asynchronously in each pixel. As an instance of
event-based camera, Dynamic and Active-pixel Vision Sensor (DAVIS) combines a
standard camera and an event-based camera. However, traditional models could
not deal with the event stream asynchronously. To analyze the event stream
asynchronously, most existing approaches accumulate events within a certain
time interval and treat the accumulated events as a synchronous frame, which
wastes the intensity change information and weakens the advantages of DAVIS.
Therefore, in this paper, we present the first neural asynchronous approach to
process event stream for event-based camera. Our method asynchronously extracts
dynamic information from events by leveraging previous motion and critical
features of gray-scale frames. To our best knowledge, this is the first neural
asynchronous method to analyze event stream through a novel deep neural
network. Extensive experiments demonstrate that our proposed model achieves
remarkable improvements against the state-of-the-art baselines.

该论文介绍了一种基于神经网络的异步处理 Event-based Camera 事件流的方法，利用前一时刻的动态信息和灰度帧的关键特征，提取事件中的动态信息，相较现有同类研究结果表现优异。

使用异步 “事件” 进行运动估计更好

Asynchronous "Events" are Better For Motion Estimation

Social media has become a popular means for people to consume news.
Meanwhile, it also enables the wide dissemination of fake news, i.e., news with
intentionally false information, which brings significant negative effects to
the society. Thus, fake news detection is attracting increasing attention.
However, fake news detection is a non-trivial task, which requires multi-source
information such as news content, social context, and dynamic information.
First, fake news is written to fool people, which makes it difficult to detect
fake news simply based on news contents. In addition to news contents, we need
to explore social contexts such as user engagements and social behaviors. For
example, a credible user's comment that "this is a fake news" is a strong
signal for detecting fake news. Second, dynamic information such as how fake
news and true news propagate and how users' opinions toward news pieces are
very important for extracting useful patterns for (early) fake news detection
and intervention. Thus, comprehensive datasets which contain news content,
social context, and dynamic information could facilitate fake news propagation,
detection, and mitigation; while to the best of our knowledge, existing
datasets only contains one or two aspects. Therefore, in this paper, to
facilitate fake news related researches, we provide a fake news data repository
FakeNewsNet, which contains two comprehensive datasets that includes news
content, social context, and dynamic information. We present a comprehensive
description of datasets collection, demonstrate an exploratory analysis of this
data repository from different perspectives, and discuss the benefits of
FakeNewsNet for potential applications on fake news study on social media.

通过提供包含新闻内容、社会背景和动态信息的数据集，本文旨在促进对社交媒体上假新闻的传播、检测和干预的研究，强调了假新闻检测的复杂性，并结合多源信息提出了可能的解决方案。