Understanding and predicting the properties of inorganic materials is crucial for accelerating advancements in materials science and driving applications in energy, electronics, and beyond. Integrating material structure data with language-based information through multi-modal large language models (LLMs) offers great potential to support these efforts by enhancing human-AI interaction. However, a key challenge lies in integrating atomic structures at full resolution into LLMs. In this work, we introduce MatterChat, a versatile structure-aware multi-modal LLM that unifies material structural data and textual inputs into a single cohesive model. MatterChat employs a bridging module to effectively align a pretrained machine learning interatomic potential with a pretrained LLM, reducing training costs and enhancing flexibility. Our results demonstrate that MatterChat significantly improves performance in material property prediction and human-AI interaction, surpassing general-purpose LLMs such as GPT-4. We also demonstrate its usefulness in applications such as more advanced scientific reasoning and step-by-step material synthesis.

本研究针对无机材料性质理解与预测面临的挑战，提出了一种新颖的多模态大型语言模型MatterChat。它通过融合材料结构数据与文本信息，显著提升了材料性质预测的性能及人机交互的效果，超越了通用大型语言模型如GPT-4，在科学推理和材料合成等应用中展现了其潜在价值。

MatterChat：用于材料科学的多模态大型语言模型