The widespread application of multimodal machine learning models like GPT-4
has revolutionized various research fields including computer vision and
natural language processing. However, its implementation in materials
informatics remains underexplored, despite the presence of materials data
across diverse modalities, such as composition and structure. The effectiveness
of machine learning models trained on large calculated datasets depends on the
accuracy of calculations, while experimental datasets often have limited data
availability and incomplete information. This paper introduces a novel approach
to multimodal machine learning in materials science via composition-structure
bimodal learning. The proposed COmposition-Structure Bimodal Network (COSNet)
is designed to enhance learning and predictions of experimentally measured
materials properties that have incomplete structure information. Bimodal
learning significantly reduces prediction errors across distinct materials
properties including Li conductivity in solid electrolyte, band gap, refractive
index, dielectric constant, energy, and magnetic moment, surpassing
composition-only learning methods. Furthermore, we identified that data
augmentation based on modal availability plays a pivotal role in the success of
bimodal learning.

该研究介绍了一种基于组成 - 结构双模态学习的多模态机器学习方法，用于提高实验测得的材料性质的学习和预测，同时通过数据增强技术，显著减小了材料性质的预测误差。