The use of machine learning methods for predicting the properties of
crystalline materials encounters significant challenges, primarily related to
input encoding, output versatility, and interpretability. Here, we introduce
CrystalBERT, an adaptable transformer-based framework with novel structure that
integrates space group, elemental, and unit cell information. The method's
adaptability lies not only in its ability to seamlessly combine diverse
features but also in its capability to accurately predict a wide range of
physically important properties, including topological properties,
superconducting transition temperatures, dielectric constants, and more.
CrystalBERT also provides insightful physical interpretations regarding the
features that most significantly influence the target properties. Our findings
indicate that space group and elemental information are more important for
predicting topological and superconducting properties, in contrast to some
properties that primarily depend on the unit cell information. This underscores
the intricate nature of topological and superconducting properties. By
incorporating all these features, we achieve a high accuracy of 91% in
topological classification, surpassing prior studies and identifying previously
misclassified topological materials, further demonstrating the effectiveness of
our model.

本研究介绍了一种基于 Transformer 的适应性框架 CrystalBERT，该框架整合了空间群、元素和晶胞信息，能够准确预测各种物理重要性质，包括拓扑性质和超导过渡温度等，同时提供具有洞察力的对目标物性影响最显著的特征的物理解释。通过结合各种特征，模型在拓扑分类中实现了 91% 的高准确率，超过以往研究，并识别了先前被错误分类的拓扑材料，进一步证明了我们模型的有效性。