Large language models excel in many human-language tasks but often falter in
highly specialized domains like scholarly astronomy. To bridge this gap, we
introduce AstroLLaMA, a 7-billion-parameter model fine-tuned from LLaMA-2 using
over 300,000 astronomy abstracts from arXiv. Optimized for traditional causal
language modeling, AstroLLaMA achieves a 30% lower perplexity than Llama-2,
showing marked domain adaptation. Our model generates more insightful and
scientifically relevant text completions and embedding extraction than
state-of-the-arts foundation models despite having significantly fewer
parameters. AstroLLaMA serves as a robust, domain-specific model with broad
fine-tuning potential. Its public release aims to spur astronomy-focused
research, including automatic paper summarization and conversational agent
development.

AstroLLaMA 是一个从 LLaMA-2 微调而来的 70 亿参数模型，通过使用 arXiv 上的 30 万篇天文学摘要进行微调，优化传统的因果语言建模，在困惑度上比 Llama-2 低 30%。尽管参数数量显著较少，我们的模型生成的文本补全和嵌入提取比当前的基础模型更富有见解和科学相关性。AstroLLaMA 作为一个具有广泛微调潜力的稳健的领域专用模型，其公开发布旨在推动天文学研究，包括自动论文摘要和对话系统开发。