In this work we present a novel task of understanding unintentional human
activities in videos. We formalize this problem as a reasoning task under
zero-shot scenario, where given a video of an unintentional activity we want to
know why it transitioned from intentional to unintentional. We first evaluate
the effectiveness of current state-of-the-art Large Multimodal Models on this
reasoning task and observe that they suffer from hallucination. We further
propose a novel prompting technique,termed as Dream of Thoughts (DoT), which
allows the model to navigate through hallucinated thoughts to achieve better
reasoning. To evaluate the performance on this task, we also introduce three
different specialized metrics designed to quantify the models reasoning
capability. We perform our experiments on two different datasets, OOPs and
UCF-Crimes, and our findings show that DOT prompting technique is able to
outperform standard prompting, while minimizing hallucinations.

我们提出了理解视频中无意识人类活动的新任务，并将此问题形式化为零样本情况下的推理任务，我们通过评估当前最先进的大型多模型在这个推理任务上的效果发现它们存在幻觉问题，因此我们提出了一种名为 Dream of Thoughts（DoT）的新型提示技术，它允许模型在幻觉的思维中导航以实现更好的推理，为了评估模型在该任务上的性能，我们还引入了三个不同的专门度量模型推理能力的指标，通过在两个不同数据集 OOPs 和 UCF-Crimes 上进行实验，我们的研究结果表明 DoT 提示技术能够在最小化幻觉的同时优于标准提示方式。