Accurate and comprehensive material databases extracted from research papers
are critical for materials science and engineering but require significant
human effort to develop. In this paper we present a simple method of extracting
materials data from full texts of research papers suitable for quickly
developing modest-sized databases. The method requires minimal to no coding,
prior knowledge about the extracted property, or model training, and provides
high recall and almost perfect precision in the resultant database. The method
is fully automated except for one human-assisted step, which typically requires
just a few hours of human labor. The method builds on top of natural language
processing and large general language models but can work with almost any such
model. The language models GPT-3/3.5, bart and DeBERTaV3 are evaluated here for
comparison. We provide a detailed detailed analysis of the methods performance
in extracting bulk modulus data, obtaining up to 90% precision at 96% recall,
depending on the amount of human effort involved. We then demonstrate the
methods broader effectiveness by developing a database of critical cooling
rates for metallic glasses.

本文主要介绍了一种从研究论文全文中提取材料数据以建立数据库的简单方法，该方法基于自然语言处理和大型通用语言模型，无需编程或模型培训等专业知识，可实现高召回率和几乎完美精度。作者还评估了多种语言模型（GPT-3/3.5，bart 和 DeBERTaV3），并在提取体积模量数据方面取得了良好的表现。最后，作者还通过开发金属玻璃的临界冷却速率数据库展示了该方法可以扩展到其他研究领域。

使用通用语言模型从文本中提取材料数据的灵活模型无关方法

Flexible, Model-Agnostic Method for Materials Data Extraction from Text Using General Purpose Language Models

We report the performance of DeBERTaV3 on CommonsenseQA in this report. We
simply formalize the answer selection as a text classification for DeBERTaV3.
The strong natural language inference ability of DeBERTaV3 helps its single and
ensemble model set the new (w/o external knowledge) state-of-the-art on
CommonsenseQA.

本文报告了 DeBERTaV3 在 CommonsenseQA 上的表现。我们使用文本分类方法对 DeBERTaV3 的答案选择问题进行形式化，并利用其自然语言推理能力，在不使用外部知识的情况下，使其单模型和集成模型取得了 CommonsenseQA 上的最新的最佳表现。

DeBERTaV3 对 CommonsenseQA 的解决方案

Solution of DeBERTaV3 on CommonsenseQA

This paper presents a new pre-trained language model, DeBERTaV3, which
improves the original DeBERTa model by replacing mask language modeling (MLM)
with replaced token detection (RTD), a more sample-efficient pre-training task.
Our analysis shows that vanilla embedding sharing in ELECTRA hurts training
efficiency and model performance. This is because the training losses of the
discriminator and the generator pull token embeddings in different directions,
creating the "tug-of-war" dynamics. We thus propose a new gradient-disentangled
embedding sharing method that avoids the tug-of-war dynamics, improving both
training efficiency and the quality of the pre-trained model. We have
pre-trained DeBERTaV3 using the same settings as DeBERTa to demonstrate its
exceptional performance on a wide range of downstream natural language
understanding (NLU) tasks. Taking the GLUE benchmark with eight tasks as an
example, the DeBERTaV3 Large model achieves a 91.37% average score, which is
1.37% over DeBERTa and 1.91% over ELECTRA, setting a new state-of-the-art
(SOTA) among the models with a similar structure. Furthermore, we have
pre-trained a multi-lingual model mDeBERTa and observed a larger improvement
over strong baselines compared to English models. For example, the mDeBERTa
Base achieves a 79.8% zero-shot cross-lingual accuracy on XNLI and a 3.6%
improvement over XLM-R Base, creating a new SOTA on this benchmark. We have
made our pre-trained models and inference code publicly available at
this https URL

提出使用更高效的预训练任务 Replaced Token Detection (RTD) 替代 Mask Language Modeling (MLM) 的新预训练语言模型 DeBERTaV3，并采用新的梯度解耦嵌入共享方法以提高训练效率和质量。在众多的下游 NLU 任务中，DeBERTaV3 表现出较其他同类结构的模型更高的正确率。