Efficiently extracting data from tables in the scientific literature is
pivotal for building large-scale databases. However, the tables reported in
materials science papers exist in highly diverse forms; thus, rule-based
extractions are an ineffective approach. To overcome this challenge, we present
MaTableGPT, which is a GPT-based table data extractor from the materials
science literature. MaTableGPT features key strategies of table data
representation and table splitting for better GPT comprehension and filtering
hallucinated information through follow-up questions. When applied to a vast
volume of water splitting catalysis literature, MaTableGPT achieved an
extraction accuracy (total F1 score) of up to 96.8%. Through comprehensive
evaluations of the GPT usage cost, labeling cost, and extraction accuracy for
the learning methods of zero-shot, few-shot and fine-tuning, we present a
Pareto-front mapping where the few-shot learning method was found to be the
most balanced solution owing to both its high extraction accuracy (total F1
score>95%) and low cost (GPT usage cost of 5.97 US dollars and labeling cost of
10 I/O paired examples). The statistical analyses conducted on the database
generated by MaTableGPT revealed valuable insights into the distribution of the
overpotential and elemental utilization across the reported catalysts in the
water splitting literature.

利用 MaTableGPT 从材料科学文献中高效提取表格数据，成功实现了 96.8% 的提取准确率，并展示了几乎无错误的学习方法的成本分析和 Pareto 前沿映射，为水分解催化领域的研究提供了宝贵的洞察。

MaTableGPT: 材料科学文献中基于 GPT 的表格数据提取器

MaTableGPT: GPT-based Table Data Extractor from Materials Science  Literature

Natural Language Processing (NLP), a cornerstone field within artificial
intelligence, has been increasingly utilized in the field of materials science
literature. Our study conducts a reproducibility analysis of two pioneering
works within this domain: "Machine-learned and codified synthesis parameters of
oxide materials" by Kim et al., and "Unsupervised word embeddings capture
latent knowledge from materials science literature" by Tshitoyan et al. We aim
to comprehend these studies from a reproducibility perspective, acknowledging
their significant influence on the field of materials informatics, rather than
critiquing them. Our study indicates that both papers offered thorough
workflows, tidy and well-documented codebases, and clear guidance for model
evaluation. This makes it easier to replicate their results successfully and
partially reproduce their findings. In doing so, they set commendable standards
for future materials science publications to aspire to. However, our analysis
also highlights areas for improvement such as to provide access to training
data where copyright restrictions permit, more transparency on model
architecture and the training process, and specifications of software
dependency versions. We also cross-compare the word embedding models between
papers, and find that some key differences in reproducibility and
cross-compatibility are attributable to design choices outside the bounds of
the models themselves. In summary, our study appreciates the benchmark set by
these seminal papers while advocating for further enhancements in research
reproducibility practices in the field of NLP for materials science. This
balance of understanding and continuous improvement will ultimately propel the
intersecting domains of NLP and materials science literature into a future of
exciting discoveries.

我们的研究对两篇杰出的物质科学文献领域的先驱作品进行了重现性分析：金和安迪的 “氧化物材料机器学习和编码合成参数” 和西托扬的 “无监督词嵌入从物质科学文献中捕捉潜在知识”。我们的研究发现这两篇论文的工作流程严谨，代码可复现，获得了明确的模型评估指导；然而，还存在改进的空间，如尽可能提供训练数据的访问权限，提供更多关于模型架构和训练过程的透明度，以及规定软件依赖版本。