The pursuit of novel alloys tailored to specific requirements poses
significant challenges for researchers in the field. This underscores the
importance of developing predictive techniques for essential physical
properties of alloys based on their chemical composition and processing
parameters. This study introduces AlloyBERT, a transformer encoder-based model
designed to predict properties such as elastic modulus and yield strength of
alloys using textual inputs. Leveraging the pre-trained RoBERTa encoder model
as its foundation, AlloyBERT employs self-attention mechanisms to establish
meaningful relationships between words, enabling it to interpret human-readable
input and predict target alloy properties. By combining a tokenizer trained on
our textual data and a RoBERTa encoder pre-trained and fine-tuned for this
specific task, we achieved a mean squared error (MSE) of 0.00015 on the Multi
Principal Elemental Alloys (MPEA) data set and 0.00611 on the Refractory Alloy
Yield Strength (RAYS) dataset. This surpasses the performance of shallow
models, which achieved a best-case MSE of 0.00025 and 0.0076 on the MPEA and
RAYS datasets respectively. Our results highlight the potential of language
models in material science and establish a foundational framework for
text-based prediction of alloy properties that does not rely on complex
underlying representations, calculations, or simulations.

使用文本输入预测合金的弹性模量和屈服强度，AlloyBERT 是一种基于变压器编码器的模型，使用 RoBERTa 作为基础，通过自注意机制建立单词之间的关联，实现对合金属性的预测。