The development of artificial intelligence systems capable of understanding and reasoning about complex real-world scenarios is a significant challenge. In this work we present a novel approach to enhance and exploit LLM reactive capability to address complex problems and interpret deeply contextual real-world meaning. We introduce a method and a tool for creating a multimodal, knowledge-augmented formal representation of meaning that combines the strengths of large language models with structured semantic representations. Our method begins with an image input, utilizing state-of-the-art large language models to generate a natural language description. This description is then transformed into an Abstract Meaning Representation (AMR) graph, which is formalized and enriched with logical design patterns, and layered semantics derived from linguistic and factual knowledge bases. The resulting graph is then fed back into the LLM to be extended with implicit knowledge activated by complex heuristic learning, including semantic implicatures, moral values, embodied cognition, and metaphorical representations. By bridging the gap between unstructured language models and formal semantic structures, our method opens new avenues for tackling intricate problems in natural language understanding and reasoning.

本研究解决了人工智能系统在理解和推理复杂现实场景中的挑战。我们提出了一种新方法，通过图形和逻辑设计模式增强大规模语言模型的反应能力，实现多模态、知识增强的意义形式化表达。此研究的关键发现是，结合语言模型和结构化语义表示，能够改善自然语言理解和推理的复杂问题处理能力。

基于神经符号的图增强 grounded 世界模型