Recent advances in visual reasoning (VR), particularly with the aid of Large
Vision-Language Models (VLMs), show promise but require access to large-scale
datasets and face challenges such as high computational costs and limited
generalization capabilities. Compositional visual reasoning approaches have
emerged as effective strategies; however, they heavily rely on the commonsense
knowledge encoded in Large Language Models (LLMs) to perform planning,
reasoning, or both, without considering the effect of their decisions on the
visual reasoning process, which can lead to errors or failed procedures. To
address these challenges, we introduce HYDRA, a multi-stage dynamic
compositional visual reasoning framework designed for reliable and
incrementally progressive general reasoning. HYDRA integrates three essential
modules: a planner, a Reinforcement Learning (RL) agent serving as a cognitive
controller, and a reasoner. The planner and reasoner modules utilize an LLM to
generate instruction samples and executable code from the selected instruction,
respectively, while the RL agent dynamically interacts with these modules,
making high-level decisions on selection of the best instruction sample given
information from the historical state stored through a feedback loop. This
adaptable design enables HYDRA to adjust its actions based on previous feedback
received during the reasoning process, leading to more reliable reasoning
outputs and ultimately enhancing its overall effectiveness. Our framework
demonstrates state-of-the-art performance in various VR tasks on four different
widely-used datasets.

HYDRA 是一个多阶段动态组合视觉推理框架，利用大规模数据集和语言模型进行可靠且递增进化的推理，表现出了在各种视觉推理任务上的最新性能。

HYDRA：用于动态组合视觉推理的超级智能体

HYDRA: A Hyper Agent for Dynamic Compositional Visual Reasoning

Extracting object-level representations for downstream reasoning tasks is an
emerging area in AI. Learning object-centric representations in an unsupervised
setting presents multiple challenges, a key one being binding an arbitrary
number of object instances to a specialized object slot. Recent object-centric
representation methods like Slot Attention utilize iterative attention to learn
composable representations with dynamic inference level binding but fail to
achieve specialized slot level binding. To address this, in this paper we
propose Unsupervised Conditional Slot Attention using a novel Probabilistic
Slot Dictionary (PSD). We define PSD with (i) abstract object-level property
vectors as key and (ii) parametric Gaussian distribution as its corresponding
value. We demonstrate the benefits of the learnt specific object-level
conditioning distributions in multiple downstream tasks, namely object
discovery, compositional scene generation, and compositional visual reasoning.
We show that our method provides scene composition capabilities and a
significant boost in a few shot adaptability tasks of compositional visual
reasoning, while performing similarly or better than slot attention in object
discovery tasks

本研究提出了一种基于无监督条件化槽注意力和概率槽字典（PSD）的方法，利用抽象的物体属性向量作为关键字、参数化高斯分布作为相应值，来学习特定的物体级别条件分布，并在多个下游任务中展示了其在物体发现、组合场景生成和组合视觉推理方面的优势。在物体发现任务中，我们的方法表现出相似或更好的性能，并在组合视觉推理的少样本适应性任务中显著提高了场景构成能力。